From gianmail at gmail.com Fri Jul 1 07:11:07 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Fri, 1 Jul 2011 14:11:07 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> Message-ID: Jed, Barry, thanks for your answers. I've tried to implement Jed's solution as it was looking faster to try. This is what I came up with for two different orders, FENOrder and FWNOrder respectively 283 KSP ksp; 284 PC pc; 285 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); CHKERRQ(ierr); 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); 295 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); CHKERRQ(ierr); 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); I don't know if this is what you intended... is there a way to make KSP recompute the factorization with the different ordering without destroying and recreating the KSP? Concerning Barry's option, js it possible to use the PETSc ilu factorization function (and eventually the ones provided by external libraries) inside my PCSHELL, passing it the various ordering I will need? If so, what's the best way to access it? Thanks again Gianluca On 20 June 2011 22:20, Barry Smith wrote: > > ?I would suggest implementing your own PCSHELL that contains the four factored matrices and orders. The PCSetUp() you write would compute the orderings and call the Mat factorization routines. The PCApply() you write would do the appropriate applications of the triangular solves. The PCILU in PETSc is just for one ordering and factorization so is not a good starting point for your algorithm. > > ? Barry > > On Jun 20, 2011, at 8:48 AM, Jed Brown wrote: > >> On Mon, Jun 20, 2011 at 16:26, Gianluca Meneghello wrote: >> My goal is to perform some relaxation sweeps for each ordering at each >> level of a multigrid process, probably using ksptype PREONLY and >> pctype ILU. >> Is that possible? >> >> Yes, but you would need to perform a separate factorization for each ordering. Also, the orderings that are not aligned with the grid in memory will have poor performance. This is a general problem with using different orderings. >> >> Is there the equivalent of -pc_sor_its with ILU >> (-pc_ilu_its maybe)? >> >> -ksp_type richardson is the same thing, put it inside -pc_type bjacobi for -pc_sor_lits. (-pc_sor_its is just a lower overhead way to do that cycling). >> >> >> I also have the problem that in order to build the ordering I would >> need to have access to a structure containing some grid informations, >> and it seems I cannot pass that structure to the YourOrdering function >> you suggested. >> >> You can PetscObjectCompose() your structure to the Mat. You might need PetscContainerCreate() to wrap your struct. >> >> >> I guess a solution for me could be to build the IS from an external >> function (not used by petsc) and then attach them directly to the mat >> structure. I also guess that the one to use are the ones at line 36 of >> >> No, those slots are not public. Use PetscObjectCompose(). > > -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From jed at 59A2.org Fri Jul 1 07:43:33 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 1 Jul 2011 07:43:33 -0500 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> Message-ID: On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello wrote: > I've tried to implement Jed's solution as it was looking faster to > try. This is what I came up with for two different orders, FENOrder > and FWNOrder respectively > > 283 KSP ksp; > 284 PC pc; > 285 > 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); > 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); > 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); > CHKERRQ(ierr); > 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); > 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); > 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); > 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); > 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); > 295 > 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); > 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); > 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); > CHKERRQ(ierr); > 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); > 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); > 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); > 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); > 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); > It looks like you are putting all of this in PCApply_YourShell(). I suggest making struct YourShellContext { PC fenpc,fwnpc; Vec work; PetscBool issetup; }; Then pass a pointer to this thing in when you create the PCShell. Then something like PCSetUp_YourShell(PC pc) { struct YourShellContext *ctx; Mat A,B; MatStructure mstr; const char *prefix; char iprefix[256]; PCShellGetContext(pc,&ctx); PCGetOptionsPrefix(pc,&prefix); if (!ctx->issetup) { ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); PCCreate(comm,&ctx->fenpc); PCSetType(ctx->fenpc,PCILU); PCFactorSetFill(ctx->fenpc,1); PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); snprintf(iprefix,sizeof iprefix,"%sfen",prefix); PCSetOptionsPrefix(ctx->fenpc,iprefix); PCSetFromOptions(ctx->fenpc); /* Same as above for ctx->fwnpc */ } PCGetOperators(pc,&A,&B,&mstr); PCSetOperators(ctx->fenpc,A,B,mstr); PCSetOperators(ctx->fwnpc,A,B,mstr); PCSetUp(ctx->fenpc); PCSetUp(ctx->fwnpc); ctx->issetup = PETSC_TRUE; } Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { ... PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); PCApply(ctx->fenpc,X,Y); /* apply first ordering */ VecScale(Y,-1.0); MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ PCApply(ctx->fwnpc,ctx->work,Y); } > I don't know if this is what you intended... is there a way to make > KSP recompute the factorization with the different ordering without > destroying and recreating the KSP? > Store both as in the code above. > > Concerning Barry's option, js it possible to use the PETSc ilu > factorization function (and eventually the ones provided by external > libraries) inside my PCSHELL, passing it the various ordering I will > need? If so, what's the best way to access it? > Does the code above answer this question? -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Fri Jul 1 17:19:37 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sat, 02 Jul 2011 00:19:37 +0200 Subject: [petsc-users] Best way to solve eqns for Runge Kutta 3rd order schemes Message-ID: <4E0E47F9.4000004@gmail.com> Hi, I need to solve the momentum eqns of my CFD code which uses the RK3 scheme. I need to solve the x and y momentum eqns 3 times per time step. Hence, for each sub time step, I have 2 linear eqns for x and y, which I solve using PETSc. The off-diagonal elements of the matrix are different for each substep. But the off-diagonal elements for the same substep is the same for different time. ie Off-diagonal elements (total 4) of the matrix of substep 1 at t=1 is the same as that of substep 1 at t=2,3.... Similarly, off-diagonal elements of the matrix of substep 2 at t=1 is the same as that of substep 2 at t=2,3.... Moreover, the off-diagonal elements of the matrix of substep 1,2 and 3 differs by a fixed factor ie if not considering the diagonal elements, matrix A(substep 1) == k1*A(substep 2) = k2*A(substep 3), where k1, k2 are constants However, the diagonal elements of the matrix changes all the time, between substeps and between time steps. The RHS vector also changes all the time. I am not too sure what is the best way (most efficient, fastest) to solve these linear equations. Currently, I write in the matrix at each substep. I doubt this is the best. I am thinking of: 1. Instead of creating 2 matrices for x and y, I create 6, 2 for each substep. The off-diagonal elements are always the same. I only use MatDiagonalSet to change the diagonal elements for each matrix at different time steps or 2. I only create 2 matrices. At each substep, I multiply the matrices by a constant, so that off-diagonal elements at one substep is equal to that of another. Then I use MatDiagonalSet to do the same as above. Which is a better mtd? Or is there other better mtds? My main concern is to save time. Moreover, how often do I have to do call: call KSPCreate(MPI_COMM_WORLD,ksp_semi_x,ierr) call KSPGetPC(ksp_semi_x,pc_semi_x,ierr) ksptype=KSPBCGS call KSPSetType(ksp_semi_x,ksptype,ierr) call KSPSetFromOptions(ksp_semi_x,ierr) tol=1.e-5 call KSPSetTolerances(ksp_semi_x,tol,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,ierr) Thank you very much for the help! -- Yours sincerely, TAY wee-beng From jed at 59A2.org Fri Jul 1 17:40:00 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 1 Jul 2011 17:40:00 -0500 Subject: [petsc-users] Best way to solve eqns for Runge Kutta 3rd order schemes In-Reply-To: <4E0E47F9.4000004@gmail.com> References: <4E0E47F9.4000004@gmail.com> Message-ID: On Fri, Jul 1, 2011 at 17:19, TAY wee-beng wrote: > Currently, I write in the matrix at each substep. I doubt this is the best. > Profile (perhaps with -log_summary, you may have to register a stage around your assembly function) and see how much time is spent in assembly. If it's not too much, then don't worry about it. If it is too much of the run time, use a fine-grained profiler to see where the time is being spent. Sometimes you can adjust mesh traversal to get better memory performance, or to remove other inefficiencies. If it is still too much, think about how you can use blocking, for example, to solve each velocity component at once (fewer independent assemblies, fewer synchronization points). If it is still too much, then perhaps consider alternatives below. I am thinking of: > > 1. Instead of creating 2 matrices for x and y, I create 6, 2 for each > substep. The off-diagonal elements are always the same. I only use > MatDiagonalSet to change the diagonal elements for each matrix at different > time steps > This uses a lot more memory. > > or > > 2. I only create 2 matrices. At each substep, I multiply the matrices by a > constant, so that off-diagonal elements at one substep is equal to that of > another. Then I use MatDiagonalSet to do the same as above. > If you can compute the diagonal without traversing the mesh, then this makes sense. -------------- next part -------------- An HTML attachment was scrubbed... URL: From goldmann at itp.uni-bremen.de Mon Jul 4 10:24:39 2011 From: goldmann at itp.uni-bremen.de (Elias Goldmann) Date: Mon, 04 Jul 2011 17:24:39 +0200 Subject: [petsc-users] Number of decimals in Viewer Message-ID: <4E11DB37.2070000@itp.uni-bremen.de> Hi, is there any possibility to change the number of decimals written to the file while using PetscRealView ? Thank you, Elias From jedbrown at mcs.anl.gov Mon Jul 4 10:27:59 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 4 Jul 2011 10:27:59 -0500 Subject: [petsc-users] Number of decimals in Viewer In-Reply-To: <4E11DB37.2070000@itp.uni-bremen.de> References: <4E11DB37.2070000@itp.uni-bremen.de> Message-ID: On Mon, Jul 4, 2011 at 10:24, Elias Goldmann wrote: > is there any possibility to change the number of decimals written to the > file > while using PetscRealView ? > There is not currently a way to set the format/precision of ASCII numeric output, but it is generally better to use binary files for anything other than casual inspection by a human. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vanharen at nrg.eu Mon Jul 4 12:09:17 2011 From: vanharen at nrg.eu (Haren, S.W. van (Steven)) Date: Mon, 4 Jul 2011 19:09:17 +0200 Subject: [petsc-users] Increasing parallel speed-up References: Message-ID: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19D@NRGPEX.nrgnet.intra> Dear all, I have been working with PETSc for a few months now. At the moment I am using it to solve a simple 2D diffusion problem on a Cartesian grid using a 5-point difference stencil. Setting up the matrix is relatively easy and the parallel performance of this part is acceptable. However, solving the matrix with one of the ksp solvers (Conjugate Gradient method with ILU(0) preconditioning) gives poor parallel performance for the following settings: - number of unknowns ~ 2 million - 1, 2 and 4 processors (quad core CPU) - using -log_summary and -ksp_view for timing purposes The speed up that I observe for solving the matrix is around 20%, i.e., when doubling the number of processors the solution time drops only with a factor of 1.2. I observe that during solving there is no swapping from internal memory to the hard disk, and the core loading for each processor is around 100%. I tried some ways to improve speed up, like configuring with --with-debugging=0 and first running a small problem before solving the actual problem (packing problem), but this hardly improves the parallel performance of the matrix solve. My questions are therefore: - do I have too high expectations of a quad core CPU? Should I move to better hardware, and test on more processors, or larger problems (more unknowns)? - do I need to change configure settings to increase parallel speed-up for the solver? - any other tips? Kind regards, Steven -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 3705 bytes Desc: not available URL: From bsmith at mcs.anl.gov Mon Jul 4 12:21:16 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 4 Jul 2011 12:21:16 -0500 Subject: [petsc-users] Number of decimals in Viewer In-Reply-To: References: <4E11DB37.2070000@itp.uni-bremen.de> Message-ID: <8302C6CC-5335-4BD3-A24A-40B2D4822959@mcs.anl.gov> You can simply edit the PetscRealView() src/sys/error/err.c function which is very simple and add more digits On Jul 4, 2011, at 10:27 AM, Jed Brown wrote: > On Mon, Jul 4, 2011 at 10:24, Elias Goldmann wrote: > is there any possibility to change the number of decimals written to the file > while using PetscRealView ? > > There is not currently a way to set the format/precision of ASCII numeric output, but it is generally better to use binary files for anything other than casual inspection by a human. From jedbrown at mcs.anl.gov Mon Jul 4 12:24:56 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 4 Jul 2011 12:24:56 -0500 Subject: [petsc-users] Increasing parallel speed-up In-Reply-To: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19D@NRGPEX.nrgnet.intra> References: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19D@NRGPEX.nrgnet.intra> Message-ID: On Mon, Jul 4, 2011 at 12:09, Haren, S.W. van (Steven) wrote: > one of the ksp solvers (Conjugate Gradient method with ILU(0) > preconditioning) gives poor parallel performance for the > We need to identify how much the poor scaling is due to the preconditioner changing (e.g. block Jacobi with ILU(0)) such that more iterations are needed versus memory bandwidth. Run with -ksp_monitor or -ksp_converged_reason to see the iterations. You can try -pc_type asm (or algebraic multigrid using third-party libraries) to improve the iteration count. If you want help seeing what's going on, send -log_summary output for each case. > following settings: > > - number of unknowns ~ 2 million > - 1, 2 and 4 processors (quad core CPU) > What kind? In particular, what memory bus and how many channels? Sparse matrix kernels are overwhelmingly limited by memory performance, so extra cores do very little good unless the memory system is very good (or the matrix fits in cache). -------------- next part -------------- An HTML attachment was scrubbed... URL: From memmett at unc.edu Mon Jul 4 12:25:54 2011 From: memmett at unc.edu (Matthew Emmett) Date: Mon, 4 Jul 2011 13:25:54 -0400 Subject: [petsc-users] petsc4py DA with dof>1 confusion Message-ID: Hi everyone, I am trying to use petsc4py in a fairly straight forward finite volume shallow-water code (the PDE has three unknowns). I have a structured grid and am using a DA to communicate ghost cells. When I use dof=1, everything works as I would expect. However, when I try to use dof=3, I am confused by the results. I have probably mis-understood something -- any suggestions would be appreciated. For example (see attached script), first I create the DA ('dof' and 'width' are defined): da = PETSc.DA().create(dim=2, sizes=[8, 8], dof=dof, boundary_type='periodic', stencil_width=width, stencil_type='box') Next, I create global and local vectors: gvec = da.createGlobalVec() lvec = da.createLocalVector() Then, put something into the local vectors, and scatter: with da.getVecArray(lvec) as va: if dof == 1: va.array[width:-width,width:-width] = MPI.COMM_WORLD.rank+1 else: va.array[width:-width,width:-width,0] = MPI.COMM_WORLD.rank+1 da.localToGlobal(lvec, gvec) Finally, get the local vector again and print: da.globalToLocal(gvec, lvec) with da.getVecArray(lvec) as va: if dof == 1: print va.array[:,:] else: print va.array[:,:,0] With 'dof=1' I get the following on the second processor: [[ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.] [ 1. 2. 2. 2. 2. 1.]] With 'dof=3' I get the following on the second processor: [[ 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 2. 0.] [ 0. 0. 0. 0. 2. 0.] [ 0. 0. 0. 2. 2. 0.] [ 0. 0. 0. 2. 2. 0.] [ 0. 0. 0. 2. 2. 0.] [ 0. 0. 0. 2. 2. 0.] [ 0. 0. 0. 2. 2. 2.] [ 0. 0. 0. 2. 2. 2.] [ 0. 0. 0. 0. 0. 2.]] I was expecting to see the same as the 'dof=1' case, since I am setting (and printing) va.array[:,:,0]. Again, any suggestions would be appreciated. Thanks, Matt PS, thanks to the developers of PETSc and petsc4py for all your hard work. I had the pleasure of meeting Jed and Matt at KAUST recently. -------------- next part -------------- A non-text attachment was scrubbed... Name: test-da.py Type: text/x-python Size: 841 bytes Desc: not available URL: From dalcinl at gmail.com Mon Jul 4 13:17:30 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 4 Jul 2011 15:17:30 -0300 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: On 4 July 2011 14:25, Matthew Emmett wrote: > Hi everyone, > > I am trying to use petsc4py in a fairly straight forward finite volume > shallow-water code (the PDE has three unknowns). I have a structured > grid and am using a DA to communicate ghost cells. ?When I use dof=1, > everything works as I would expect. ?However, when I try to use dof=3, > I am confused by the results. ?I have probably mis-understood > something -- any suggestions would be appreciated. > > For example (see attached script), first I create the DA ('dof' and > 'width' are defined): > > ?da = PETSc.DA().create(dim=2, sizes=[8, 8], dof=dof, > ? ? ? ? ? ? ? ? ? ? ? ? boundary_type='periodic', > ? ? ? ? ? ? ? ? ? ? ? ? stencil_width=width, > ? ? ? ? ? ? ? ? ? ? ? ? stencil_type='box') > > Next, I create global and local vectors: > > ?gvec = da.createGlobalVec() > ?lvec = da.createLocalVector() > > Then, put something into the local vectors, and scatter: > > ?with da.getVecArray(lvec) as va: > ? ?if dof == 1: > ? ? ?va.array[width:-width,width:-width] = MPI.COMM_WORLD.rank+1 > ? ?else: > ? ? ?va.array[width:-width,width:-width,0] = MPI.COMM_WORLD.rank+1 > ?da.localToGlobal(lvec, gvec) > > Finally, get the local vector again and print: > > ?da.globalToLocal(gvec, lvec) > ?with da.getVecArray(lvec) as va: > ? ?if dof == 1: > ? ? ?print va.array[:,:] > ? ?else: > ? ? ?print va.array[:,:,0] > > With 'dof=1' I get the following on the second processor: > > [[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.] > ?[ 1. ?2. ?2. ?2. ?2. ?1.]] > > With 'dof=3' I get the following on the second processor: > > [[ 0. ?0. ?0. ?0. ?0. ?0.] > ?[ 0. ?0. ?0. ?0. ?2. ?0.] > ?[ 0. ?0. ?0. ?0. ?2. ?0.] > ?[ 0. ?0. ?0. ?2. ?2. ?0.] > ?[ 0. ?0. ?0. ?2. ?2. ?0.] > ?[ 0. ?0. ?0. ?2. ?2. ?0.] > ?[ 0. ?0. ?0. ?2. ?2. ?0.] > ?[ 0. ?0. ?0. ?2. ?2. ?2.] > ?[ 0. ?0. ?0. ?2. ?2. ?2.] > ?[ 0. ?0. ?0. ?0. ?0. ?2.]] > > I was expecting to see the same as the 'dof=1' case, since I am > setting (and printing) va.array[:,:,0]. > > > Again, any suggestions would be appreciated. ?Thanks, > Matt > > PS, thanks to the developers of PETSc and petsc4py for all your hard > work. ?I had the pleasure of meeting Jed and Matt at KAUST recently. > Mmm, it seems that this functionality is broken for the case of boundary_type='periodic'. I'll take a look. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From dalcinl at gmail.com Mon Jul 4 13:52:59 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 4 Jul 2011 15:52:59 -0300 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: On 4 July 2011 15:17, Lisandro Dalcin wrote: > On 4 July 2011 14:25, Matthew Emmett wrote: >> Hi everyone, >> >> I am trying to use petsc4py in a fairly straight forward finite volume >> shallow-water code (the PDE has three unknowns). I have a structured >> grid and am using a DA to communicate ghost cells. ?When I use dof=1, >> everything works as I would expect. ?However, when I try to use dof=3, >> I am confused by the results. ?I have probably mis-understood >> something -- any suggestions would be appreciated. >> >> For example (see attached script), first I create the DA ('dof' and >> 'width' are defined): >> >> ?da = PETSc.DA().create(dim=2, sizes=[8, 8], dof=dof, >> ? ? ? ? ? ? ? ? ? ? ? ? boundary_type='periodic', >> ? ? ? ? ? ? ? ? ? ? ? ? stencil_width=width, >> ? ? ? ? ? ? ? ? ? ? ? ? stencil_type='box') >> >> Next, I create global and local vectors: >> >> ?gvec = da.createGlobalVec() >> ?lvec = da.createLocalVector() >> >> Then, put something into the local vectors, and scatter: >> >> ?with da.getVecArray(lvec) as va: >> ? ?if dof == 1: >> ? ? ?va.array[width:-width,width:-width] = MPI.COMM_WORLD.rank+1 >> ? ?else: >> ? ? ?va.array[width:-width,width:-width,0] = MPI.COMM_WORLD.rank+1 >> ?da.localToGlobal(lvec, gvec) >> >> Finally, get the local vector again and print: >> >> ?da.globalToLocal(gvec, lvec) >> ?with da.getVecArray(lvec) as va: >> ? ?if dof == 1: >> ? ? ?print va.array[:,:] >> ? ?else: >> ? ? ?print va.array[:,:,0] >> >> With 'dof=1' I get the following on the second processor: >> >> [[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.] >> ?[ 1. ?2. ?2. ?2. ?2. ?1.]] >> >> With 'dof=3' I get the following on the second processor: >> >> [[ 0. ?0. ?0. ?0. ?0. ?0.] >> ?[ 0. ?0. ?0. ?0. ?2. ?0.] >> ?[ 0. ?0. ?0. ?0. ?2. ?0.] >> ?[ 0. ?0. ?0. ?2. ?2. ?0.] >> ?[ 0. ?0. ?0. ?2. ?2. ?0.] >> ?[ 0. ?0. ?0. ?2. ?2. ?0.] >> ?[ 0. ?0. ?0. ?2. ?2. ?0.] >> ?[ 0. ?0. ?0. ?2. ?2. ?2.] >> ?[ 0. ?0. ?0. ?2. ?2. ?2.] >> ?[ 0. ?0. ?0. ?0. ?0. ?2.]] >> >> I was expecting to see the same as the 'dof=1' case, since I am >> setting (and printing) va.array[:,:,0]. >> >> >> Again, any suggestions would be appreciated. ?Thanks, >> Matt >> >> PS, thanks to the developers of PETSc and petsc4py for all your hard >> work. ?I had the pleasure of meeting Jed and Matt at KAUST recently. >> > > Mmm, it seems that this functionality is broken for the case of > boundary_type='periodic'. I'll take a look. > No, sorry... It has nothing to do with periodicity... It is actually a C/Fortran ordering issue I need to fix... In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , However, I think that for Python we should follow a more C-ish indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is counter-intuitive to C (and likely Python) programmers ... What do others think about this? -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From memmett at unc.edu Mon Jul 4 14:19:25 2011 From: memmett at unc.edu (Matthew Emmett) Date: Mon, 4 Jul 2011 15:19:25 -0400 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 2:52 PM, Lisandro Dalcin wrote: > No, sorry... It has nothing to do with periodicity... It is actually a > C/Fortran ordering issue I need to fix... Ah, I see. Thanks for looking into this so quickly. Let me know if there is anything I can do on my end to help. I will poke around the code for petsc4py, but you obviously know it better than I do. > In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , > However, I think that for Python we should follow a more C-ish > indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is > A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is > counter-intuitive to C (and likely Python) programmers ... > > What do others think about this? I think A[x,y,z,dof] is probably the most intuitive for Python. Matt From vanharen at nrg.eu Mon Jul 4 14:32:16 2011 From: vanharen at nrg.eu (Haren, S.W. van (Steven)) Date: Mon, 4 Jul 2011 21:32:16 +0200 Subject: [petsc-users] Increasing parallel speed-up References: Message-ID: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19F@NRGPEX.nrgnet.intra> Thank you for you reply Jed. I will take a look at the preconditioners, to see if I can increase the scaling. CPU is an Intel i7 q720, just a standard laptop CPU. Regards, Steven --------------------------- Date: Mon, 4 Jul 2011 12:24:56 -0500 From: Jed Brown Subject: Re: [petsc-users] Increasing parallel speed-up To: PETSc users list Message-ID: Content-Type: text/plain; charset="utf-8" On Mon, Jul 4, 2011 at 12:09, Haren, S.W. van (Steven) wrote: > one of the ksp solvers (Conjugate Gradient method with ILU(0) > preconditioning) gives poor parallel performance for the > We need to identify how much the poor scaling is due to the preconditioner changing (e.g. block Jacobi with ILU(0)) such that more iterations are needed versus memory bandwidth. Run with -ksp_monitor or -ksp_converged_reason to see the iterations. You can try -pc_type asm (or algebraic multigrid using third-party libraries) to improve the iteration count. If you want help seeing what's going on, send -log_summary output for each case. > following settings: > > - number of unknowns ~ 2 million > - 1, 2 and 4 processors (quad core CPU) > What kind? In particular, what memory bus and how many channels? Sparse matrix kernels are overwhelmingly limited by memory performance, so extra cores do very little good unless the memory system is very good (or the matrix fits in cache). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4863 bytes Desc: not available URL: From aron.ahmadia at kaust.edu.sa Mon Jul 4 14:33:02 2011 From: aron.ahmadia at kaust.edu.sa (Aron Ahmadia) Date: Mon, 4 Jul 2011 22:33:02 +0300 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: Hi Matt! We deal with this same issue in PyClaw/PetClaw, I think Amal could do a much better job describing the approach (or copying a relevant section from her Master's thesis) we take to avoid copying, but the idea is to follow the native PETSc ordering with interleaved degrees-of-freedom to keep the most compatibility with the other DA calls. A On Mon, Jul 4, 2011 at 10:19 PM, Matthew Emmett wrote: > On Mon, Jul 4, 2011 at 2:52 PM, Lisandro Dalcin wrote: > > No, sorry... It has nothing to do with periodicity... It is actually a > > C/Fortran ordering issue I need to fix... > > Ah, I see. Thanks for looking into this so quickly. Let me know if > there is anything I can do on my end to help. I will poke around the > code for petsc4py, but you obviously know it better than I do. > > > In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , > > However, I think that for Python we should follow a more C-ish > > indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is > > A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is > > counter-intuitive to C (and likely Python) programmers ... > > > > What do others think about this? > > I think A[x,y,z,dof] is probably the most intuitive for Python. > > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From memmett at unc.edu Mon Jul 4 14:48:14 2011 From: memmett at unc.edu (Matthew Emmett) Date: Mon, 4 Jul 2011 15:48:14 -0400 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: Hey Aaron, Thanks for the tip! I'll play around with the indexing and reshaping. Matt On Mon, Jul 4, 2011 at 3:33 PM, Aron Ahmadia wrote: > Hi Matt! > We deal with this same issue in PyClaw/PetClaw, I think Amal could do a much > better job describing the approach (or copying a relevant section from her > Master's thesis) we take to avoid copying, but the idea is to follow the > native PETSc ordering with interleaved degrees-of-freedom to keep the most > compatibility with the other DA calls. > A > On Mon, Jul 4, 2011 at 10:19 PM, Matthew Emmett wrote: >> >> On Mon, Jul 4, 2011 at 2:52 PM, Lisandro Dalcin wrote: >> > No, sorry... It has nothing to do with periodicity... It is actually a >> > C/Fortran ordering issue I need to fix... >> >> Ah, I see. ?Thanks for looking into this so quickly. ?Let me know if >> there is anything I can do on my end to help. ?I will poke around the >> code for petsc4py, but you obviously know it better than I do. >> >> > In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , >> > However, I think that for Python we should follow a more C-ish >> > indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is >> > A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is >> > counter-intuitive to C (and likely Python) programmers ... >> > >> > What do others think about this? >> >> I think A[x,y,z,dof] is probably the most intuitive for Python. >> >> Matt > > From aron.ahmadia at kaust.edu.sa Mon Jul 4 14:53:04 2011 From: aron.ahmadia at kaust.edu.sa (Aron Ahmadia) Date: Mon, 4 Jul 2011 22:53:04 +0300 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: Matt, Also, in case you are looking for an example of how it is done in PyClaw: https://github.com/clawpack/pyclaw/tree/master/apps/shallow/2d Shallow-water is not one of the automatically verified examples, but we'll be happy to help you if you have any questions about the code. A On Mon, Jul 4, 2011 at 10:48 PM, Matthew Emmett wrote: > Hey Aaron, > > Thanks for the tip! I'll play around with the indexing and reshaping. > > Matt > > On Mon, Jul 4, 2011 at 3:33 PM, Aron Ahmadia > wrote: > > Hi Matt! > > We deal with this same issue in PyClaw/PetClaw, I think Amal could do a > much > > better job describing the approach (or copying a relevant section from > her > > Master's thesis) we take to avoid copying, but the idea is to follow the > > native PETSc ordering with interleaved degrees-of-freedom to keep the > most > > compatibility with the other DA calls. > > A > > On Mon, Jul 4, 2011 at 10:19 PM, Matthew Emmett wrote: > >> > >> On Mon, Jul 4, 2011 at 2:52 PM, Lisandro Dalcin > wrote: > >> > No, sorry... It has nothing to do with periodicity... It is actually a > >> > C/Fortran ordering issue I need to fix... > >> > >> Ah, I see. Thanks for looking into this so quickly. Let me know if > >> there is anything I can do on my end to help. I will poke around the > >> code for petsc4py, but you obviously know it better than I do. > >> > >> > In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , > >> > However, I think that for Python we should follow a more C-ish > >> > indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is > >> > A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is > >> > counter-intuitive to C (and likely Python) programmers ... > >> > > >> > What do others think about this? > >> > >> I think A[x,y,z,dof] is probably the most intuitive for Python. > >> > >> Matt > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Mon Jul 4 15:46:13 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 4 Jul 2011 17:46:13 -0300 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: On 4 July 2011 16:19, Matthew Emmett wrote: > On Mon, Jul 4, 2011 at 2:52 PM, Lisandro Dalcin wrote: >> No, sorry... It has nothing to do with periodicity... It is actually a >> C/Fortran ordering issue I need to fix... > > Ah, I see. ?Thanks for looking into this so quickly. ?Let me know if > there is anything I can do on my end to help. ?I will poke around the > code for petsc4py, but you obviously know it better than I do. > >> In Fortran 90, it seems you index a DA Vec array as A[dof,x,y,z]... , >> However, I think that for Python we should follow a more C-ish >> indexing A[x,y,z,dof]. Or we could do it like PETSc in C, that is >> A[z,y,x,dof] (wich is the transpose of the Fortran way) but it is >> counter-intuitive to C (and likely Python) programmers ... >> >> What do others think about this? > > I think A[x,y,z,dof] is probably the most intuitive for Python. > OK. I pushed a fix solving the issue. As a consequence, the underlying NumPy array do have a mixed ordering (no C nor Fortran) where the latest dim varies fastest, then first, second, third. So, you have to be very careful about accessing the underlying array (I mean, doing va.array[...] in your previous code), I would recomend you against that, and code like this instead: with da.getVecArray(lvec) as va: if dof == 1: va[:,:] = rank+1 else: va[:,:,0] = rank+1 da.localToGlobal(lvec, gvec) da.globalToLocal(gvec, lvec) with da.getVecArray(lvec) as va: if dof == 1: print va[:,:] else: print va[:,:,0] -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From knepley at gmail.com Mon Jul 4 17:15:13 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 4 Jul 2011 22:15:13 +0000 Subject: [petsc-users] Increasing parallel speed-up In-Reply-To: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19F@NRGPEX.nrgnet.intra> References: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D19F@NRGPEX.nrgnet.intra> Message-ID: On Mon, Jul 4, 2011 at 7:32 PM, Haren, S.W. van (Steven) wrote: > Thank you for you reply Jed. > > I will take a look at the preconditioners, to see if I can increase the > scaling. > > CPU is an Intel i7 q720, just a standard laptop CPU. > As Jed points out, you will see very little speedup here due to the quite poor memory subsystem. Intel rarely points out that this setup is great for factoring, but lousy for large swaths of computational science: http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers Matt > Regards, > > Steven > > > > --------------------------- > Date: Mon, 4 Jul 2011 12:24:56 -0500 > From: Jed Brown > Subject: Re: [petsc-users] Increasing parallel speed-up > To: PETSc users list > Message-ID: > > > Content-Type: text/plain; charset="utf-8" > > On Mon, Jul 4, 2011 at 12:09, Haren, S.W. van (Steven) >wrote: > > > one of the ksp solvers (Conjugate Gradient method with ILU(0) > > preconditioning) gives poor parallel performance for the > > > > We need to identify how much the poor scaling is due to the preconditioner > changing (e.g. block Jacobi with ILU(0)) such that more iterations are > needed versus memory bandwidth. Run with -ksp_monitor or > -ksp_converged_reason to see the iterations. You can try -pc_type asm (or > algebraic multigrid using third-party libraries) to improve the iteration > count. > > If you want help seeing what's going on, send -log_summary output for each > case. > > > > following settings: > > > > - number of unknowns ~ 2 million > > - 1, 2 and 4 processors (quad core CPU) > > > > What kind? In particular, what memory bus and how many channels? Sparse > matrix kernels are overwhelmingly limited by memory performance, so extra > cores do very little good unless the memory system is very good (or the > matrix fits in cache). > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From memmett at unc.edu Mon Jul 4 17:23:22 2011 From: memmett at unc.edu (Matthew Emmett) Date: Mon, 4 Jul 2011 18:23:22 -0400 Subject: [petsc-users] petsc4py DA with dof>1 confusion In-Reply-To: References: Message-ID: Great, thanks Lisandro. Everything is working nicely for me now. Matt From bsmith at mcs.anl.gov Mon Jul 4 21:44:46 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 4 Jul 2011 21:44:46 -0500 Subject: [petsc-users] PETSc recommended visualization packages Message-ID: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> What are the recommended visualization packages for use with PETSc (for example making movies of contour plots and isosurfaces) and what are the recommended data formats to use to save Vecs for visualization? Thanks Barry From jedbrown at mcs.anl.gov Mon Jul 4 22:06:10 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 4 Jul 2011 22:06:10 -0500 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> Message-ID: On Mon, Jul 4, 2011 at 21:44, Barry Smith wrote: > What are the recommended visualization packages for use with PETSc (for > example making movies of contour plots and isosurfaces) and what are the > recommended data formats to use to save Vecs for visualization? VisIt and ParaView are the primary choices. In my experience, VisIt is a more complete system, but ParaView is easier to install and probably easier to get started with. The VTK XML or binary formats are the easiest to get started with. See src/ts/examples/tutorials/ex14.c for an example of writing an XML file containing two fields on different grids (2D and 3D, structured, but deformed) in parallel. All file formats are horrible non-extensible crud, usually grown out of a particular application with structured grids of low-order/non-exotic basis functions on unstructured grids. They tend not to retain enough semantic information to reconstruct a state for further computations and also work for visualization without needing additional application-provided information. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen.wornom at inria.fr Tue Jul 5 02:25:33 2011 From: stephen.wornom at inria.fr (Stephen Wornom) Date: Tue, 05 Jul 2011 09:25:33 +0200 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> Message-ID: <4E12BC6D.2080801@inria.fr> Jed Brown wrote: > On Mon, Jul 4, 2011 at 21:44, Barry Smith > wrote: > > What are the recommended visualization packages for use with PETSc > (for example making movies of contour plots and isosurfaces) and > what are the recommended data formats to use to save Vecs for > visualization? > > > VisIt Any links to VisIt? Goggle produced nothing. Thanks, Stephen > and ParaView are the primary choices. In my experience, VisIt is a > more complete system, but ParaView is easier to install and probably > easier to get started with. > > The VTK XML or binary formats are the easiest to get started with. See > src/ts/examples/tutorials/ex14.c for an example of writing an XML file > containing two fields on different grids (2D and 3D, structured, but > deformed) in parallel. All file formats are horrible non-extensible > crud, usually grown out of a particular application with structured > grids of low-order/non-exotic basis functions on unstructured grids. > They tend not to retain enough semantic information to reconstruct a > state for further computations and also work for visualization without > needing additional application-provided information. -- stephen.wornom at inria.fr 2004 route des lucioles - BP93 Sophia Antipolis 06902 CEDEX Tel: 04 92 38 50 54 Fax: 04 97 15 53 51 -------------- next part -------------- A non-text attachment was scrubbed... Name: stephen_wornom.vcf Type: text/x-vcard Size: 160 bytes Desc: not available URL: From goldmann at itp.uni-bremen.de Tue Jul 5 02:42:58 2011 From: goldmann at itp.uni-bremen.de (Elias Goldmann) Date: Tue, 05 Jul 2011 09:42:58 +0200 Subject: [petsc-users] Number of decimals in Viewer Message-ID: <4E12C082.3080106@itp.uni-bremen.de> Thank you both for the replies. I write a few real eigenvalues to a file using PetscRealView. Do you really think going binary is better here? Why would that be the case? Kind regards, Elias Date: Mon, 4 Jul 2011 12:21:16 -0500 From: Barry Smith Subject: Re: [petsc-users] Number of decimals in Viewer To: PETSc users list Message-ID:<8302C6CC-5335-4BD3-A24A-40B2D4822959 at mcs.anl.gov> Content-Type: text/plain; charset=us-ascii You can simply edit the PetscRealView() src/sys/error/err.c function which is very simple and add more digits On Jul 4, 2011, at 10:27 AM, Jed Brown wrote: > > On Mon, Jul 4, 2011 at 10:24, Elias Goldmann wrote: > > is there any possibility to change the number of decimals written to the file > > while using PetscRealView ? > > > > There is not currently a way to set the format/precision of ASCII numeric output, but it is generally better to use binary files for anything other than casual inspection by a human. From ivoroghair at gmail.com Tue Jul 5 03:55:46 2011 From: ivoroghair at gmail.com (Ivo Roghair) Date: Tue, 5 Jul 2011 10:55:46 +0200 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <4E12BC6D.2080801@inria.fr> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <4E12BC6D.2080801@inria.fr> Message-ID: https://wci.llnl.gov/codes/visit/ This one. I agree the name is too generic to find it with just visit. I always use "visit llnl". 2011/7/5 Stephen Wornom : > Jed Brown wrote: >> >> On Mon, Jul 4, 2011 at 21:44, Barry Smith > > wrote: >> >> ? ?What are the recommended visualization packages for use with PETSc >> ? ?(for example making movies of contour plots and isosurfaces) and >> ? ?what are the recommended data formats to use to save Vecs for >> ? ?visualization? >> >> >> VisIt > > Any links to VisIt? Goggle produced nothing. > Thanks, > Stephen > >> and ParaView are the primary choices. In my experience, VisIt is a more >> complete system, but ParaView is easier to install and probably easier to >> get started with. >> >> The VTK XML or binary formats are the easiest to get started with. See >> src/ts/examples/tutorials/ex14.c for an example of writing an XML file >> containing two fields on different grids (2D and 3D, structured, but >> deformed) in parallel. All file formats are horrible non-extensible crud, >> usually grown out of a particular application with structured grids of >> low-order/non-exotic basis functions on unstructured grids. They tend not to >> retain enough semantic information to reconstruct a state for further >> computations and also work for visualization without needing additional >> application-provided information. > > > -- > stephen.wornom at inria.fr > 2004 route des lucioles - BP93 > Sophia Antipolis > 06902 CEDEX > > Tel: 04 92 38 50 54 > Fax: 04 97 15 53 51 > > From jedbrown at mcs.anl.gov Tue Jul 5 07:47:47 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 5 Jul 2011 07:47:47 -0500 Subject: [petsc-users] Number of decimals in Viewer In-Reply-To: <4E12C082.3080106@itp.uni-bremen.de> References: <4E12C082.3080106@itp.uni-bremen.de> Message-ID: On Tue, Jul 5, 2011 at 02:42, Elias Goldmann wrote: > I write a few real eigenvalues to a file using PetscRealView. Do you really > think going binary is better here? Why would that be the case? This is entirely reasonable unless your results are very nearly as accurate as machine precision, in which case using text may lose a couple low bits in a platform-dependent way. Maybe it would be worth adding something like: PetscViewerASCIIPushRealFormat(viewer,"% 20.12e"); -------------- next part -------------- An HTML attachment was scrubbed... URL: From yxliuwm at gmail.com Tue Jul 5 08:01:28 2011 From: yxliuwm at gmail.com (Yixun Liu) Date: Tue, 5 Jul 2011 09:01:28 -0400 Subject: [petsc-users] Start multiple Jobs from one process Message-ID: <9AAFCC81380B4B4889907519ACBFC7EA@yliuPC> Hi, I have a PETSc application and I can start it using the command: mpiexec -n 4 app.exe I would like to know if I can start the four processes in one process. Thanks. Yixun -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Jul 5 08:19:08 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 5 Jul 2011 08:19:08 -0500 Subject: [petsc-users] Start multiple Jobs from one process In-Reply-To: <9AAFCC81380B4B4889907519ACBFC7EA@yliuPC> References: <9AAFCC81380B4B4889907519ACBFC7EA@yliuPC> Message-ID: 2011/7/5 Yixun Liu > I have a PETSc application and I can start it using the command: > mpiexec -n 4 app.exe > > I would like to know if I can start the four processes in one process. > Yes, but you would have to use the MPI-2 dynamic process management. It will take some effort and I don't recommend it unless you're really sure that's what you want. -------------- next part -------------- An HTML attachment was scrubbed... URL: From skolb at rocketmail.com Tue Jul 5 09:04:08 2011 From: skolb at rocketmail.com (Stefan Kolb) Date: Tue, 5 Jul 2011 16:04:08 +0200 Subject: [petsc-users] A Beginner Question Message-ID: <201107051604.08545.skolb@rocketmail.com> Hi all, i have written a littel test program, to check how PETSc stores a matrix across two processes. static char help[] = "PETSc test"; #include "stdio.h" #include "petscksp.h" int main(int argc,char **args) { Mat A; PetscInt N=100,M=100, row_start = 0, column_start = 0, row_end = 0, column_end = 0, rank = 0, local_rows = 0, local_columns = 0,nproc=0; PetscErrorCode ierr; PetscScalar value =0.; PetscInitialize(&argc,&args,(char *)0,help); MPI_Comm_rank(PETSC_COMM_WORLD,&rank); MPI_Comm_size(PETSC_COMM_WORLD,&nproc); ierr = PetscPrintf(PETSC_COMM_WORLD,"Number of Processes %d\n",nproc);CHKERRQ(ierr); ierr = MatCreate(PETSC_COMM_WORLD,&A); CHKERRQ(ierr); ierr = MatSetSizes(A,PETSC_DECIDE, PETSC_DECIDE,N,M); CHKERRQ(ierr); ierr = MatSetFromOptions(A);CHKERRQ(ierr); ierr = MatGetOwnershipRange(A,&row_start,&row_end);CHKERRQ(ierr); ierr = MatGetOwnershipRangeColumn(A,&column_start,&column_end);CHKERRQ(ierr); ierr = MatGetLocalSize(A,&local_rows,&local_columns);CHKERRQ(ierr); printf("process=%d \t row_start=%d \t row_end=%d \t column_start=%d \t column_end=%d \t local_rows=%d \t local_columns=%d\n",rank,row_start,row_end,column_start,column_end,local_rows,local_columns); ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatDestroy(A); CHKERRQ(ierr); ierr = PetscFinalize();CHKERRQ(ierr); return 0; } The output of the program (using 2 processes) is: Number of Processes 2 process=0 row_start=0 row_end=50 column_start=0 column_end=50 local_rows=50 local_columns=50 process=1 row_start=50 row_end=100 column_start=50 column_end=100 local_rows=50 local_columns=50 So if the output of my program is right the process with rank=0 owns a matrix with 50 rows and 50 columns in the same way the process with rank=1. This cant be right because on both processes only one half of the global matrix is stored. Wat is the error in my test programm? I tried to set the local size of the matrix by hand with the command MatSetSizes(A,50, 100,N,M) but then i get an error from PETSc. How can i tell PETSc to store the first 50 rows on process 0 and the next 50 rows on process 1? Regards, Stefan From bourdin at lsu.edu Tue Jul 5 09:58:07 2011 From: bourdin at lsu.edu (Blaise Bourdin) Date: Tue, 5 Jul 2011 16:58:07 +0200 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <4E12BC6D.2080801@inria.fr> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <4E12BC6D.2080801@inria.fr> Message-ID: Hi, Let me add to Barry's frustration. I have spent most the last week trying to render animations of reasonably large computations (about 25M 3d unstructured elements divided in up to 25,000 exodusII files). - Paraview is simple to get started with in the GUI. In theory, it is scriptable in python, but there is pretty much no documentation on the python API, which seems to be a mismatched combination of vtk calls, and several levels / revisions / versions of paraview interfaces. It is possible to generate a python trace of gui interactions, but for some reasons, some actions (image format and sizes, movies framnerates, for instance) are not recorded. - visit's gui take a little more time to master, it is also scriptable in python. The python capture is more readable, and is somewhat documented (the doc is for 1.4.0, the current version 2.3.0). Movie generation is a pain and as far as I understand not feasible inside a python script. Instead, one has to save a session, and reopen it with the -movie CL interface. - Unless you are willing to rebuild visit or paraview from source (which is highly non-trivial and poorly documented), all these package install their own python, so that you don't have access to all other packages you may have on your system. I was not able to figure out how to import paraview or visit from my own python using the binary distributions - EnSight is a good alternate choice if you can afford it. It is not open and not cheap but is quite good, reads most major format and has a pretty responsive support team. Their python scripting is well documented, but as far as I understand, it is not possible to import ensight from a python script. Instead, Ensight runs python scripts. - About data formats: hdf5 / xdmf seems very well supported, but again, human-readable documentation on xdmf is lacking. It would be really nice if one could generate the proper xdmf description and geometry from a dm of a dmmesh. Would it be also possible to interface with moab and gain access to all the file formats they support? Finally, I am skeptical about the benefit of implementing yet another file format that is incompatible with existing analysis, post processing, and mesh generation tools. Blaise > Jed Brown wrote: >> On Mon, Jul 4, 2011 at 21:44, Barry Smith > wrote: >> >> What are the recommended visualization packages for use with PETSc >> (for example making movies of contour plots and isosurfaces) and >> what are the recommended data formats to use to save Vecs for >> visualization? >> >> >> VisIt > Any links to VisIt? Goggle produced nothing. > Thanks, > Stephen > >> and ParaView are the primary choices. In my experience, VisIt is a more complete system, but ParaView is easier to install and probably easier to get started with. >> >> The VTK XML or binary formats are the easiest to get started with. See src/ts/examples/tutorials/ex14.c for an example of writing an XML file containing two fields on different grids (2D and 3D, structured, but deformed) in parallel. All file formats are horrible non-extensible crud, usually grown out of a particular application with structured grids of low-order/non-exotic basis functions on unstructured grids. They tend not to retain enough semantic information to reconstruct a state for further computations and also work for visualization without needing additional application-provided information. > > > -- > stephen.wornom at inria.fr > 2004 route des lucioles - BP93 > Sophia Antipolis > 06902 CEDEX > > Tel: 04 92 38 50 54 > Fax: 04 97 15 53 51 > > -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin From jedbrown at mcs.anl.gov Tue Jul 5 10:33:38 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 5 Jul 2011 10:33:38 -0500 Subject: [petsc-users] A Beginner Question In-Reply-To: <201107051604.08545.skolb@rocketmail.com> References: <201107051604.08545.skolb@rocketmail.com> Message-ID: On Tue, Jul 5, 2011 at 09:04, Stefan Kolb wrote: > I tried to set the local size of the matrix by hand with the command > MatSetSizes(A,50, 100,N,M) but then i get an error from PETSc. How can i > tell PETSc to store the first 50 rows on process 0 and the next 50 rows on > process 1? What you did is correct, but MatGetOwnershipRangeColumn() returns the ownership range of a vector that the matrix can be applied to. See the this man page for details on how MPIAIJ matrices are stored: http://www.mcs.anl.gov/petsc/petsc-2/snapshots/petsc-dev/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From baagaard at usgs.gov Tue Jul 5 10:53:38 2011 From: baagaard at usgs.gov (Brad Aagaard) Date: Tue, 05 Jul 2011 08:53:38 -0700 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <4E12BC6D.2080801@inria.fr> Message-ID: <4E133382.3050808@usgs.gov> Hi, I agree with Blaise's assessment of ParaView. It is good for quick and dirty visualization, but its scripting interface is very limited. MayaVi provides a GUI along with high-level API access to VTK data structures through Python. One can update/modify the data values through the API without ever writing VTK compatible files. I read my data from HDF5 files using PyTables and create VTK data objects using the API. A similar mechanism could be used for data stored using SQLite. I think some Linux and Mac packaging systems include MayaVi; I usually build from source. Regards, Brad On 07/05/2011 07:58 AM, Blaise Bourdin wrote: > Hi, > > Let me add to Barry's frustration. I have spent most the last week > trying to render animations of reasonably large computations (about > 25M 3d unstructured elements divided in up to 25,000 exodusII > files). > > - Paraview is simple to get started with in the GUI. In theory, it is > scriptable in python, but there is pretty much no documentation on > the python API, which seems to be a mismatched combination of vtk > calls, and several levels / revisions / versions of paraview > interfaces. It is possible to generate a python trace of gui > interactions, but for some reasons, some actions (image format and > sizes, movies framnerates, for instance) are not recorded. - visit's > gui take a little more time to master, it is also scriptable in > python. The python capture is more readable, and is somewhat > documented (the doc is for 1.4.0, the current version 2.3.0). Movie > generation is a pain and as far as I understand not feasible inside a > python script. Instead, one has to save a session, and reopen it with > the -movie CL interface. - Unless you are willing to rebuild visit or > paraview from source (which is highly non-trivial and poorly > documented), all these package install their own python, so that you > don't have access to all other packages you may have on your system. > I was not able to figure out how to import paraview or visit from my > own python using the binary distributions - EnSight is a good > alternate choice if you can afford it. It is not open and not cheap > but is quite good, reads most major format and has a pretty > responsive support team. Their python scripting is well documented, > but as far as I understand, it is not possible to import ensight from > a python script. Instead, Ensight runs python scripts. > > - About data formats: hdf5 / xdmf seems very well supported, but > again, human-readable documentation on xdmf is lacking. It would be > really nice if one could generate the proper xdmf description and > geometry from a dm of a dmmesh. Would it be also possible to > interface with moab and gain access to all the file formats they > support? > > Finally, I am skeptical about the benefit of implementing yet another > file format that is incompatible with existing analysis, post > processing, and mesh generation tools. > > Blaise From dalcinl at gmail.com Tue Jul 5 11:58:03 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 5 Jul 2011 13:58:03 -0300 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <4E133382.3050808@usgs.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <4E12BC6D.2080801@inria.fr> <4E133382.3050808@usgs.gov> Message-ID: On 5 July 2011 12:53, Brad Aagaard wrote: > Hi, > > I agree with Blaise's assessment of ParaView. It is good for quick and > dirty visualization, but its scripting interface is very limited. > > MayaVi provides a GUI along with high-level API access to VTK data > structures through Python. One can update/modify the data values through > the API without ever writing VTK compatible files. I read my data from > HDF5 files using PyTables and create VTK data objects using the API. A > similar mechanism could be used for data stored using SQLite. I think > some Linux and Mac packaging systems include MayaVi; I usually build > from source. > I recently added support (requires petsc4py, of course) for -ksp|snes|ts_monitor_python filename.py:function|class. Perhaps you can find this useful for "live" monitoring of simulations. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From adam1.byrd at gmail.com Tue Jul 5 12:17:13 2011 From: adam1.byrd at gmail.com (Adam Byrd) Date: Tue, 5 Jul 2011 13:17:13 -0400 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: Message-ID: I have working code that produces the correct answer now, thank you. One (hopefully) final question, though. The solution is actually transposed. What causes this? Presumably I can use MatMatSolveTranspose to get around this, but there's no man page and I want to be sure of what will happen in all cases. Respectfully, Adam On Thu, Jun 30, 2011 at 4:18 PM, Hong Zhang wrote: > Add > ierr = MatGetOrdering(testMat,MATORDERING_ND,&isrow,&iscol);CHKERRQ(ierr); > before the line > ierr = MatFactorInfoInitialize(&luinfo);CHKERRQ(ierr); > > Somehow, your matrix is numerically singular. With > MATORDERING_NATURAL > I get > [0]PETSC ERROR: Detected zero pivot in LU factorization > > even using MATDENSE matrix format, which calls lapack. > With MATORDERING_ND, I get useless inverseMat. > > The modified code I used is attached. > > Hong > > On Thu, Jun 30, 2011 at 2:30 PM, Adam Byrd wrote: > > I'm trying to work through what I need to do, again by practicing with a > > small scale random problem. The general order of events seems to be: > create > > a matrix, fill it, assemble it, factor it, then one can use solvers with > it. > > When I use MatLUFactor on my matrix before using it with a solver I get > this > > error: > > [0]PETSC ERROR: --------------------- Error Message > > ------------------------------------ > > [0]PETSC ERROR: Null argument, when expecting valid pointer! > > [0]PETSC ERROR: Null Object: Parameter # 1! > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 > > CDT 2011 > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > [0]PETSC ERROR: See docs/index.html for manual pages. > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: ./test on a osx-gnu-c named Macintosh-3.local by adambyrd > > Thu Jun 30 15:27:30 2011 > > [0]PETSC ERROR: Libraries linked from > > /Users/adambyrd/soft/petsc-3.1-p8/osx-gnu-cpp/lib > > [0]PETSC ERROR: Configure run at Tue Jun 28 12:56:55 2011 > > [0]PETSC ERROR: Configure options PETSC_ARCH=osx-gnu-cpp > --with-fc=gfortran > > -download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex > > --with-clanguage=c++ > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: ISInvertPermutation() line 209 in > > src/vec/is/interface/index.c > > [0]PETSC ERROR: MatLUFactorSymbolic_SeqAIJ() line 306 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatLUFactorSymbolic() line 2534 in > > src/mat/interface/matrix.c > > [0]PETSC ERROR: MatLUFactor_SeqAIJ() line 945 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatLUFactor() line 2417 in src/mat/interface/matrix.c > > [0]PETSC ERROR: main() line 62 in WDtest.cpp > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0[unset]: > > aborting job: > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0 > > I don't understand what I'm doing wrong. > > Respectfully, > > Adam > > On Wed, Jun 29, 2011 at 1:26 AM, Matthew Knepley > wrote: > >> > >> On Tue, Jun 28, 2011 at 10:20 PM, Adam Byrd > wrote: > >>> > >>> Matt, > >>> > >>> Alright, that means I need to continue learning how to use > >>> MatSetValues(). With my 6x6 example I tried filling it with four 3x3 > sub > >>> matrices, but when I do that I get the error 'sum of local sizes 12 > does not > >>> equal global size.' I had 4 processors each calling MatSetValues for > their > >>> own 3x3. Graphically, I arranged the nodes 0 1 > >>> > >>> > 2 3 > >>> where process 0 had global rows 0-2 and global columns 0-2; process 1 > had > >>> 0-2, 3-5; process 2 had 3-5, 0-2; and process 3 had 3-5, 3-5. From the > >>> documentation, I think this should be correct, but I'm not sure. Also, > which > >>> format would you recommend for storing the matrix? > >> > >> 1) With any error, send the Entire error message. > >> 2) PETSc matrices are divided by rows, not rows and columns, see the > >> manual section. Rows & columns only makes sense for dense matrices > >> 3) You can still set arbitrary blocks no matter how the matrix is > divided > >> 4) The error means you tried to set both local and global dimensions, > and > >> they do not add up correctly. Just set the global dimensions > >> Matt > >> > >>> > >>> Jack, > >>> > >>> I'm a summer intern just getting started with this project, so I don't > >>> know all the details yet (I can ask though). I know I need to find the > >>> Green's function which will involve the trace of the inverted > Hamiltonian, > >>> as well as the rest of the matrix. I have inquired about avoiding the > >>> inversion altogether, but my instructor doesn't believe there is a way > >>> around it. Once I've worked through the math I want to explore other > options > >>> though. > >>> > >>> Respectfully, > >>> Adam > >>> > >>> On Tue, Jun 28, 2011 at 6:08 PM, Matthew Knepley > >>> wrote: > >>>> > >>>> On Tue, Jun 28, 2011 at 5:01 PM, Adam Byrd > wrote: > >>>>> > >>>>> Actually, it's quite sparse. In the 3600x3600 there are only just 4 > >>>>> nonzero entries in each row. This means it's 99.9% empty. My smaller > 6x6 > >>>>> example is dense, but it's only practice building and manipulating > matrices. > >>>> > >>>> Ah, then its easy. Just call MatSetValues() with each block. Then use > >>>> MUMPS to do a sparse direct solve. > >>>> Matt > >>>> > >>>>> > >>>>> Respectfully, > >>>>> Adam > >>>>> > >>>>> On Tue, Jun 28, 2011 at 5:55 PM, Matthew Knepley > >>>>> wrote: > >>>>>> > >>>>>> It sounds like you have a dense matrix (from your example). Is this > >>>>>> true? If so, you should use Elemental (on Google Code). > >>>>>> Thanks, > >>>>>> Matt > >>>>>> > >>>>>> On Tue, Jun 28, 2011 at 8:55 AM, Adam Byrd > >>>>>> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> I'm rather new to PETSc and trying to work out the best way to > create > >>>>>>> and fill a large sparse matrix distributed over many processors. > Currently, > >>>>>>> my goal is to create a 3600x3600 matrix in units of 12x12 blocks > with > >>>>>>> several blocks on any given node. I'd like to create the matrix in > such a > >>>>>>> way that each node only holds the information in it's handful of > blocks and > >>>>>>> not the entire matrix. Eventually, this matrix is to be inverted (I > know, > >>>>>>> inversion should be avoided, but as this is a Hamiltonian matrix > from which > >>>>>>> I need the Green's function, I'm unaware of a way to forgo carrying > out the > >>>>>>> inversion). Additionally, the values will be changed slightly and > the matrix > >>>>>>> will be repeatedly inverted. It's structure will remain the same. > In order > >>>>>>> to learn how to do this is I am starting with a small 6x6 matrix > broken into > >>>>>>> four 3x3 blocks and distributed one block per node. I've been able > to create > >>>>>>> a local 3x3 matrix on each node, with it's own values, and with the > global > >>>>>>> row/column IDs correctly set to [0, 1, 2] or [3, 4, 5] depending on > where > >>>>>>> the block is in the matrix. My problem manifests when I try to > create the > >>>>>>> larger matrix from the individual smaller ones. When the matrix is > >>>>>>> constructed I'm trying to use MatSetValues and having each node > pass in it's > >>>>>>> 3x3 block. I end up with an error that the sum of local lengths > 12x12 does > >>>>>>> not match the global length 6x6. It appears as though this is from > passing > >>>>>>> in four 3x3s and the program interpreting that as a 12x12 instead > of as a > >>>>>>> 6x6 with the blocks in a grid. > >>>>>>> My question is then: is it possible to fill a matrix as a grid of > >>>>>>> blocks, or can I only fill it in groups of rows or columns? Also, > am I > >>>>>>> approaching this problem the correct way, or are there more > efficient ways > >>>>>>> of building this matrix with the ultimate goal of inverting it? > >>>>>>> I have included my copy of a modified example if it helps. I do > >>>>>>> apologize if this is answered somewhere in the documentation, I > have been > >>>>>>> unable to find a solution. > >>>>>>> Respectfully, > >>>>>>> Adam > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> What most experimenters take for granted before they begin their > >>>>>> experiments is infinitely more interesting than any results to which > their > >>>>>> experiments lead. > >>>>>> -- Norbert Wiener > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> What most experimenters take for granted before they begin their > >>>> experiments is infinitely more interesting than any results to which > their > >>>> experiments lead. > >>>> -- Norbert Wiener > >>> > >> > >> > >> > >> -- > >> What most experimenters take for granted before they begin their > >> experiments is infinitely more interesting than any results to which > their > >> experiments lead. > >> -- Norbert Wiener > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Jul 5 12:29:31 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 5 Jul 2011 17:29:31 +0000 Subject: [petsc-users] A Beginner Question In-Reply-To: <201107051604.08545.skolb@rocketmail.com> References: <201107051604.08545.skolb@rocketmail.com> Message-ID: On Tue, Jul 5, 2011 at 2:04 PM, Stefan Kolb wrote: > Hi all, > i have written a littel test program, to check how PETSc stores a matrix > across two processes. > > static char help[] = "PETSc test"; > > #include "stdio.h" > > #include "petscksp.h" > > int main(int argc,char **args) > { > Mat A; > PetscInt N=100,M=100, row_start = 0, > column_start = 0, row_end = 0, column_end = 0, rank = 0, local_rows = 0, > local_columns = 0,nproc=0; > PetscErrorCode ierr; > PetscScalar value =0.; > > PetscInitialize(&argc,&args,(char *)0,help); > > MPI_Comm_rank(PETSC_COMM_WORLD,&rank); > MPI_Comm_size(PETSC_COMM_WORLD,&nproc); > > ierr = PetscPrintf(PETSC_COMM_WORLD,"Number of Processes > %d\n",nproc);CHKERRQ(ierr); > ierr = MatCreate(PETSC_COMM_WORLD,&A); CHKERRQ(ierr); > ierr = MatSetSizes(A,PETSC_DECIDE, PETSC_DECIDE,N,M); CHKERRQ(ierr); > ierr = MatSetFromOptions(A);CHKERRQ(ierr); > ierr = MatGetOwnershipRange(A,&row_start,&row_end);CHKERRQ(ierr); > ierr = > MatGetOwnershipRangeColumn(A,&column_start,&column_end);CHKERRQ(ierr); > ierr = MatGetLocalSize(A,&local_rows,&local_columns);CHKERRQ(ierr); > > printf("process=%d \t row_start=%d \t row_end=%d \t column_start=%d > \t column_end=%d \t local_rows=%d \t > local_columns=%d\n",rank,row_start,row_end,column_start,column_end,local_rows,local_columns); > > ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > > ierr = MatDestroy(A); CHKERRQ(ierr); > > ierr = PetscFinalize();CHKERRQ(ierr); > return 0; > } > > The output of the program (using 2 processes) is: > Number of Processes 2 > process=0 row_start=0 row_end=50 column_start=0 > column_end=50 local_rows=50 local_columns=50 > process=1 row_start=50 row_end=100 column_start=50 > column_end=100 local_rows=50 local_columns=50 > > So if the output of my program is right the process with rank=0 owns a > matrix with 50 rows and 50 columns in the same way the process with rank=1. > This cant be right because on both processes only one half of the global > matrix is stored. Wat is the error in my test programm? > I tried to set the local size of the matrix by hand with the command > MatSetSizes(A,50, 100,N,M) but then i get an error from PETSc. How can i > tell PETSc to store the first 50 rows on process 0 and the next 50 rows on > process 1? > All matrix formats store by row. The column ownership is only used to determine compatibility with vector it might act on. Thanks, Matt > Regards, > Stefan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From travis.fisher at nasa.gov Tue Jul 5 14:38:12 2011 From: travis.fisher at nasa.gov (Travis C. Fisher) Date: Tue, 5 Jul 2011 15:38:12 -0400 Subject: [petsc-users] Question for time dependent problems Message-ID: <4E136824.2010405@nasa.gov> I am wondering if there is a "right way" to set up preconditioning for time dependent nonlinear problems (using precondtioned matrix free SNES). For example, I want to use implicit Runge Kutta, and freeze my preconditioner at the beginning of each step. Currently how I do this is: Problem set up: 1. Specify nonzero structure of precondtioner matrix. 2. Specify MAT_NEW_NONZERO_LOCATION_ERR and MAT_KEEP_NONZERO_PATTERN. Beginning of a time step (1st implicit stage of RK scheme): 1. Use MatZeroEntries on the preconditioner 2. Calculate and assemble preconditioning matrix 3. In the jacobian evaluation routine, use flag = SAME_NONZERO_PATTERN (I'd like it to use the nonzero structure I've already specified) Subsequent nonlinear solves in time step: 1. Use flag = SAME_PRECONDITIONER in the jacobian evaluation In this way, I only initialize the preconditioning matrix structure once. I am wondering if I have made any large mistakes here or if I am misunderstanding the character of the solver? Is there a tutorial case for this situation? I want to note that my initial nonzero structure may contain entries that are zero, but may not be zero in subsequent calculations. I am wondering what PETSC does with these during ASM/ILU preconditioning? i.e. Does it remove them from my originally specified nonzero structure? I appreciate any direction you can give me. Thanks, Travis Fisher From bsmith at mcs.anl.gov Tue Jul 5 15:35:37 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jul 2011 15:35:37 -0500 Subject: [petsc-users] Question for time dependent problems In-Reply-To: <4E136824.2010405@nasa.gov> References: <4E136824.2010405@nasa.gov> Message-ID: On Jul 5, 2011, at 2:38 PM, Travis C. Fisher wrote: > I am wondering if there is a "right way" to set up preconditioning for time dependent nonlinear problems (using precondtioned matrix free SNES). For example, I want to use implicit Runge Kutta, and freeze my preconditioner at the beginning of each step. Currently how I do this is: > > Problem set up: > 1. Specify nonzero structure of precondtioner matrix. > 2. Specify MAT_NEW_NONZERO_LOCATION_ERR and MAT_KEEP_NONZERO_PATTERN. > > Beginning of a time step (1st implicit stage of RK scheme): > 1. Use MatZeroEntries on the preconditioner Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. > 2. Calculate and assemble preconditioning matrix > 3. In the jacobian evaluation routine, use flag = SAME_NONZERO_PATTERN (I'd like it to use the nonzero structure I've already specified) > > Subsequent nonlinear solves in time step: > 1. Use flag = SAME_PRECONDITIONER in the jacobian evaluation > > In this way, I only initialize the preconditioning matrix structure once. I am wondering if I have made any large mistakes here or if I am misunderstanding the character of the solver? > > Is there a tutorial case for this situation? I want to note that my initial nonzero structure may contain entries that are zero, but may not be zero in subsequent calculations. Just make sure you put the zeros into the matrix the first time you build it and PETSc will keep the space around for the later calculations. > I am wondering what PETSC does with these during ASM/ILU preconditioning? i.e. Does it remove them from my originally specified nonzero structure? It does not remove zeros so long as you put them in. Barry > > I appreciate any direction you can give me. > > Thanks, > > Travis Fisher From bsmith at mcs.anl.gov Tue Jul 5 15:39:09 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jul 2011 15:39:09 -0500 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: Message-ID: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> MatSolve() and MatMatSolve() will not return the transpose of the solution, they will return the solution. If you want the transpose solve you can use MatSolveTranspose() for a single Vec or use MatMatSolve() for the matrix case after transposing the original matrix if you want the transpose. Barry On Jul 5, 2011, at 12:17 PM, Adam Byrd wrote: > I have working code that produces the correct answer now, thank you. One (hopefully) final question, though. The solution is actually transposed. What causes this? Presumably I can use MatMatSolveTranspose to get around this, but there's no man page and I want to be sure of what will happen in all cases. > > Respectfully, > Adam > > On Thu, Jun 30, 2011 at 4:18 PM, Hong Zhang wrote: > Add > ierr = MatGetOrdering(testMat,MATORDERING_ND,&isrow,&iscol);CHKERRQ(ierr); > before the line > ierr = MatFactorInfoInitialize(&luinfo);CHKERRQ(ierr); > > Somehow, your matrix is numerically singular. With > MATORDERING_NATURAL > I get > [0]PETSC ERROR: Detected zero pivot in LU factorization > > even using MATDENSE matrix format, which calls lapack. > With MATORDERING_ND, I get useless inverseMat. > > The modified code I used is attached. > > Hong > > On Thu, Jun 30, 2011 at 2:30 PM, Adam Byrd wrote: > > I'm trying to work through what I need to do, again by practicing with a > > small scale random problem. The general order of events seems to be: create > > a matrix, fill it, assemble it, factor it, then one can use solvers with it. > > When I use MatLUFactor on my matrix before using it with a solver I get this > > error: > > [0]PETSC ERROR: --------------------- Error Message > > ------------------------------------ > > [0]PETSC ERROR: Null argument, when expecting valid pointer! > > [0]PETSC ERROR: Null Object: Parameter # 1! > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 > > CDT 2011 > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > [0]PETSC ERROR: See docs/index.html for manual pages. > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: ./test on a osx-gnu-c named Macintosh-3.local by adambyrd > > Thu Jun 30 15:27:30 2011 > > [0]PETSC ERROR: Libraries linked from > > /Users/adambyrd/soft/petsc-3.1-p8/osx-gnu-cpp/lib > > [0]PETSC ERROR: Configure run at Tue Jun 28 12:56:55 2011 > > [0]PETSC ERROR: Configure options PETSC_ARCH=osx-gnu-cpp --with-fc=gfortran > > -download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex > > --with-clanguage=c++ > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: ISInvertPermutation() line 209 in > > src/vec/is/interface/index.c > > [0]PETSC ERROR: MatLUFactorSymbolic_SeqAIJ() line 306 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatLUFactorSymbolic() line 2534 in > > src/mat/interface/matrix.c > > [0]PETSC ERROR: MatLUFactor_SeqAIJ() line 945 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatLUFactor() line 2417 in src/mat/interface/matrix.c > > [0]PETSC ERROR: main() line 62 in WDtest.cpp > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0[unset]: > > aborting job: > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0 > > I don't understand what I'm doing wrong. > > Respectfully, > > Adam > > On Wed, Jun 29, 2011 at 1:26 AM, Matthew Knepley wrote: > >> > >> On Tue, Jun 28, 2011 at 10:20 PM, Adam Byrd wrote: > >>> > >>> Matt, > >>> > >>> Alright, that means I need to continue learning how to use > >>> MatSetValues(). With my 6x6 example I tried filling it with four 3x3 sub > >>> matrices, but when I do that I get the error 'sum of local sizes 12 does not > >>> equal global size.' I had 4 processors each calling MatSetValues for their > >>> own 3x3. Graphically, I arranged the nodes 0 1 > >>> > >>> 2 3 > >>> where process 0 had global rows 0-2 and global columns 0-2; process 1 had > >>> 0-2, 3-5; process 2 had 3-5, 0-2; and process 3 had 3-5, 3-5. >From the > >>> documentation, I think this should be correct, but I'm not sure. Also, which > >>> format would you recommend for storing the matrix? > >> > >> 1) With any error, send the Entire error message. > >> 2) PETSc matrices are divided by rows, not rows and columns, see the > >> manual section. Rows & columns only makes sense for dense matrices > >> 3) You can still set arbitrary blocks no matter how the matrix is divided > >> 4) The error means you tried to set both local and global dimensions, and > >> they do not add up correctly. Just set the global dimensions > >> Matt > >> > >>> > >>> Jack, > >>> > >>> I'm a summer intern just getting started with this project, so I don't > >>> know all the details yet (I can ask though). I know I need to find the > >>> Green's function which will involve the trace of the inverted Hamiltonian, > >>> as well as the rest of the matrix. I have inquired about avoiding the > >>> inversion altogether, but my instructor doesn't believe there is a way > >>> around it. Once I've worked through the math I want to explore other options > >>> though. > >>> > >>> Respectfully, > >>> Adam > >>> > >>> On Tue, Jun 28, 2011 at 6:08 PM, Matthew Knepley > >>> wrote: > >>>> > >>>> On Tue, Jun 28, 2011 at 5:01 PM, Adam Byrd wrote: > >>>>> > >>>>> Actually, it's quite sparse. In the 3600x3600 there are only just 4 > >>>>> nonzero entries in each row. This means it's 99.9% empty. My smaller 6x6 > >>>>> example is dense, but it's only practice building and manipulating matrices. > >>>> > >>>> Ah, then its easy. Just call MatSetValues() with each block. Then use > >>>> MUMPS to do a sparse direct solve. > >>>> Matt > >>>> > >>>>> > >>>>> Respectfully, > >>>>> Adam > >>>>> > >>>>> On Tue, Jun 28, 2011 at 5:55 PM, Matthew Knepley > >>>>> wrote: > >>>>>> > >>>>>> It sounds like you have a dense matrix (from your example). Is this > >>>>>> true? If so, you should use Elemental (on Google Code). > >>>>>> Thanks, > >>>>>> Matt > >>>>>> > >>>>>> On Tue, Jun 28, 2011 at 8:55 AM, Adam Byrd > >>>>>> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> I'm rather new to PETSc and trying to work out the best way to create > >>>>>>> and fill a large sparse matrix distributed over many processors. Currently, > >>>>>>> my goal is to create a 3600x3600 matrix in units of 12x12 blocks with > >>>>>>> several blocks on any given node. I'd like to create the matrix in such a > >>>>>>> way that each node only holds the information in it's handful of blocks and > >>>>>>> not the entire matrix. Eventually, this matrix is to be inverted (I know, > >>>>>>> inversion should be avoided, but as this is a Hamiltonian matrix from which > >>>>>>> I need the Green's function, I'm unaware of a way to forgo carrying out the > >>>>>>> inversion). Additionally, the values will be changed slightly and the matrix > >>>>>>> will be repeatedly inverted. It's structure will remain the same. In order > >>>>>>> to learn how to do this is I am starting with a small 6x6 matrix broken into > >>>>>>> four 3x3 blocks and distributed one block per node. I've been able to create > >>>>>>> a local 3x3 matrix on each node, with it's own values, and with the global > >>>>>>> row/column IDs correctly set to [0, 1, 2] or [3, 4, 5] depending on where > >>>>>>> the block is in the matrix. My problem manifests when I try to create the > >>>>>>> larger matrix from the individual smaller ones. When the matrix is > >>>>>>> constructed I'm trying to use MatSetValues and having each node pass in it's > >>>>>>> 3x3 block. I end up with an error that the sum of local lengths 12x12 does > >>>>>>> not match the global length 6x6. It appears as though this is from passing > >>>>>>> in four 3x3s and the program interpreting that as a 12x12 instead of as a > >>>>>>> 6x6 with the blocks in a grid. > >>>>>>> My question is then: is it possible to fill a matrix as a grid of > >>>>>>> blocks, or can I only fill it in groups of rows or columns? Also, am I > >>>>>>> approaching this problem the correct way, or are there more efficient ways > >>>>>>> of building this matrix with the ultimate goal of inverting it? > >>>>>>> I have included my copy of a modified example if it helps. I do > >>>>>>> apologize if this is answered somewhere in the documentation, I have been > >>>>>>> unable to find a solution. > >>>>>>> Respectfully, > >>>>>>> Adam > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> What most experimenters take for granted before they begin their > >>>>>> experiments is infinitely more interesting than any results to which their > >>>>>> experiments lead. > >>>>>> -- Norbert Wiener > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> What most experimenters take for granted before they begin their > >>>> experiments is infinitely more interesting than any results to which their > >>>> experiments lead. > >>>> -- Norbert Wiener > >>> > >> > >> > >> > >> -- > >> What most experimenters take for granted before they begin their > >> experiments is infinitely more interesting than any results to which their > >> experiments lead. > >> -- Norbert Wiener > > > > > From dalcinl at gmail.com Tue Jul 5 17:42:19 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 5 Jul 2011 19:42:19 -0300 Subject: [petsc-users] Question for time dependent problems In-Reply-To: References: <4E136824.2010405@nasa.gov> Message-ID: On 5 July 2011 17:35, Barry Smith wrote: > >> Beginning of a time step (1st implicit stage of RK scheme): >> 1. Use MatZeroEntries on the preconditioner > > ?Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. > What do you mean? The OP said that the nonzero structure was set at problem set up. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From bsmith at mcs.anl.gov Tue Jul 5 20:03:06 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jul 2011 20:03:06 -0500 Subject: [petsc-users] Question for time dependent problems In-Reply-To: References: <4E136824.2010405@nasa.gov> Message-ID: On Jul 5, 2011, at 5:42 PM, Lisandro Dalcin wrote: > On 5 July 2011 17:35, Barry Smith wrote: >> >>> Beginning of a time step (1st implicit stage of RK scheme): >>> 1. Use MatZeroEntries on the preconditioner >> >> Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. >> > > What do you mean? The OP said that the nonzero structure was set at > problem set up. Perhaps I miss understood. I interpreted that to mean the matrix was provided with the correct preallocation information (and not the nonzero structure) but yes if the nonzero structure is already set then one can call MatZeroEntries() at that point. Perhaps we should "fix" MatZeroEntries()" to not screw up preallocation information! Barry > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169 From pratik.mallya.ml at gmail.com Tue Jul 5 23:53:13 2011 From: pratik.mallya.ml at gmail.com (pratik) Date: Wed, 06 Jul 2011 10:23:13 +0530 Subject: [petsc-users] sgi mpt errors Message-ID: <4E13EA39.20205@gmail.com> Hi everyone, I am getting the following errors on testing PETSc on an SGI ALTIX cluster with sgi mpt-1.23.:(building and install was without any problems): /opt/intel/Compiler/11.1/038/bin/intel64/icc -o ex19.o -c -Wno-deprecated -g -I/home/pratikm/install/include -I/home/pratikm/install/include -I/usr/X11R6/include -I/usr/include -I/usr/lib64 -I/opt/sgi/mpt/mpt-1.23/include -D__INSDIR__=src/snes/examples/tutorials/ ex19.c /opt/intel/Compiler/11.1/038/bin/intel64/icc -Wno-deprecated -g -o ex19 ex19.o -Wl,-rpath,/home/pratikm/install/lib -L/home/pratikm/install/lib -lpetsc -L/usr/X11R6/lib64 -lX11 -Wl,-rpath,/usr/lib64 -L/usr/lib64 -lhdf5 -llapack -lblas -Wl,-rpath,/opt/sgi/mpt/mpt-1.23/lib -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -ldl -L/opt/intel/Compiler/11.1/038/lib/intel64 -L/usr/lib64/gcc/x86_64-suse-linux/4.1.2 -L/usr/x86_64-suse-linux/lib -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lgfortran -lm -lm -ldl -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -ldl /home/pratikm/install/lib/libpetsc.a(bvec2.o): In function `VecView_Seq_Binary': /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/seq/bvec2.c:401: undefined reference to `mpi_sgi_status_ignore' /home/pratikm/install/lib/libpetsc.a(pdvec.o): In function `VecView_MPI_Binary': /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/mpi/pdvec.c:459: undefined reference to `mpi_sgi_status_ignore' /home/pratikm/install/lib/libpetsc.a(gr2.o): In function `DAArrayMPIIO': /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:478: undefined reference to `mpi_sgi_status_ignore' /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:480: undefined reference to `mpi_sgi_status_ignore' /home/pratikm/install/lib/libpetsc.a(vecio.o): In function `VecLoad_Binary': /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/utils/vecio.c:275: undefined reference to `mpi_sgi_status_ignore' /home/pratikm/install/lib/libpetsc.a(mpiov.o): In function `MatGetSubMatrix_MPIAIJ_All': /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:585: undefined reference to `mpi_sgi_inplace' /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:646: undefined reference to `mpi_sgi_inplace' /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:720: undefined reference to `mpi_sgi_inplace' /home/pratikm/install/lib/libpetsc.a(mpibaij.o): In function `MatGetSeqNonzerostructure_MPIBAIJ': /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2391: undefined reference to `mpi_sgi_inplace' /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2450: undefined reference to `mpi_sgi_inplace' make[3]: [ex19] Error 1 (ignored) /bin/rm -f ex19.o Can anyone please tell me what options i need to provide when configuring so that the program works correctly? here is the configure option that i used: ./config/configure.py --prefix=/home/pratikm/install -with-cc=/opt/intel/Compiler/11.1/038/bin/intel64/icc --CFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 -xT -g -with-cxx=/opt/intel/Compiler/11.1/038/bin/intel64/icpc --CXXFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi++ -lmpi -lsma --with-mpi-include=/opt/sgi/mpt/mpt-1.23/include --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so --with-hdf5-include=/usr/include --with-hdf5-lib=/usr/lib64/libhdf5.so Thanks in advance, Pratik From vijay.m at gmail.com Wed Jul 6 00:00:34 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Wed, 6 Jul 2011 00:00:34 -0500 Subject: [petsc-users] Question for time dependent problems In-Reply-To: References: <4E136824.2010405@nasa.gov> Message-ID: Barry, >> Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. I found an earlier thread ( http://lists.mcs.anl.gov/pipermail/petsc-users/2011-April/008566.html) where you said that it would not destroy the preallocation too. So is the behavior different in the dev version ? Vijay On Jul 5, 2011 8:03 PM, "Barry Smith" wrote: > > On Jul 5, 2011, at 5:42 PM, Lisandro Dalcin wrote: > >> On 5 July 2011 17:35, Barry Smith wrote: >>> >>>> Beginning of a time step (1st implicit stage of RK scheme): >>>> 1. Use MatZeroEntries on the preconditioner >>> >>> Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. >>> >> >> What do you mean? The OP said that the nonzero structure was set at >> problem set up. > > Perhaps I miss understood. I interpreted that to mean the matrix was provided with the correct preallocation information (and not the nonzero structure) but yes if the nonzero structure is already set then one can call MatZeroEntries() at that point. > > Perhaps we should "fix" MatZeroEntries()" to not screw up preallocation information! > > Barry > > >> >> -- >> Lisandro Dalcin >> --------------- >> CIMEC (INTEC/CONICET-UNL) >> Predio CONICET-Santa Fe >> Colectora RN 168 Km 472, Paraje El Pozo >> 3000 Santa Fe, Argentina >> Tel: +54-342-4511594 (ext 1011) >> Tel/Fax: +54-342-4511169 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Jul 6 00:10:34 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 6 Jul 2011 00:10:34 -0500 (CDT) Subject: [petsc-users] sgi mpt errors In-Reply-To: <4E13EA39.20205@gmail.com> References: <4E13EA39.20205@gmail.com> Message-ID: what do you have for: nm -Ao /opt/sgi/mpt/mpt-1.23/lib/* |grep -i mpi_sgi_status_ignore > --CFLAGS=-Wno-deprecated > -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 > -xT -g > --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so Perhaps you should use: CFLAGS="-Wno-deprecated -xT" COPTFLAGS="-O3 -g" --with-mpi-lib="-L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma" Satish On Wed, 6 Jul 2011, pratik wrote: > Hi everyone, > > I am getting the following errors on testing PETSc on an SGI ALTIX cluster > with sgi mpt-1.23.:(building and install was without any problems): > > /opt/intel/Compiler/11.1/038/bin/intel64/icc -o ex19.o -c -Wno-deprecated -g > -I/home/pratikm/install/include -I/home/pratikm/install/include > -I/usr/X11R6/include -I/usr/include -I/usr/lib64 > -I/opt/sgi/mpt/mpt-1.23/include -D__INSDIR__=src/snes/examples/tutorials/ > ex19.c > /opt/intel/Compiler/11.1/038/bin/intel64/icc -Wno-deprecated -g -o ex19 > ex19.o -Wl,-rpath,/home/pratikm/install/lib -L/home/pratikm/install/lib > -lpetsc -L/usr/X11R6/lib64 -lX11 -Wl,-rpath,/usr/lib64 -L/usr/lib64 -lhdf5 > -llapack -lblas -Wl,-rpath,/opt/sgi/mpt/mpt-1.23/lib > -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -ldl > -L/opt/intel/Compiler/11.1/038/lib/intel64 > -L/usr/lib64/gcc/x86_64-suse-linux/4.1.2 -L/usr/x86_64-suse-linux/lib -limf > -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lgfortran -lm -lm -ldl -limf > -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -ldl > /home/pratikm/install/lib/libpetsc.a(bvec2.o): In function > `VecView_Seq_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/seq/bvec2.c:401: > undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(pdvec.o): In function > `VecView_MPI_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/mpi/pdvec.c:459: > undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(gr2.o): In function `DAArrayMPIIO': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:478: undefined > reference to `mpi_sgi_status_ignore' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:480: undefined > reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(vecio.o): In function `VecLoad_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/utils/vecio.c:275: > undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(mpiov.o): In function > `MatGetSubMatrix_MPIAIJ_All': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:585: > undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:646: > undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:720: > undefined reference to `mpi_sgi_inplace' > /home/pratikm/install/lib/libpetsc.a(mpibaij.o): In function > `MatGetSeqNonzerostructure_MPIBAIJ': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2391: > undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2450: > undefined reference to `mpi_sgi_inplace' > make[3]: [ex19] Error 1 (ignored) > /bin/rm -f ex19.o > > > Can anyone please tell me what options i need to provide when configuring so > that the program works correctly? > here is the configure option that i used: > > ./config/configure.py --prefix=/home/pratikm/install > -with-cc=/opt/intel/Compiler/11.1/038/bin/intel64/icc --CFLAGS=-Wno-deprecated > -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 > -xT -g -with-cxx=/opt/intel/Compiler/11.1/038/bin/intel64/icpc > --CXXFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include > -L/opt/sgi/mpt/mpt-1.23/lib -lmpi++ -lmpi -lsma > --with-mpi-include=/opt/sgi/mpt/mpt-1.23/include > --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so > --with-hdf5-include=/usr/include --with-hdf5-lib=/usr/lib64/libhdf5.so > > > Thanks in advance, > Pratik > From pratik.mallya.ml at gmail.com Wed Jul 6 03:11:04 2011 From: pratik.mallya.ml at gmail.com (pratik) Date: Wed, 06 Jul 2011 13:41:04 +0530 Subject: [petsc-users] sgi mpt errors In-Reply-To: References: <4E13EA39.20205@gmail.com> Message-ID: <4E141898.9080406@gmail.com> Hi Satish, On Wednesday 06 July 2011 10:40 AM, Satish Balay wrote: > what do you have for: > > > nm -Ao /opt/sgi/mpt/mpt-1.23/lib/* |grep -i mpi_sgi_status_ignore > > /opt/sgi/mpt/mpt-1.23/lib/libmpi_mt.so:00000000003865f0 B mpi_sgi_status_ignore /opt/sgi/mpt/mpt-1.23/lib/libmpi.so:00000000003835d0 B mpi_sgi_status_ignore >> --CFLAGS=-Wno-deprecated >> -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 >> -xT -g >> --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so >> > Perhaps you should use: > > CFLAGS="-Wno-deprecated -xT" COPTFLAGS="-O3 -g" --with-mpi-lib="-L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma" > > Satish > > Unfortunately, I am still getting the same errors. I used nm to search for the rest of the missing symbols, all of them seem to be in libmpi.so, so i don't understand why the test is still giving these errors. Thanks once again, Pratik > On Wed, 6 Jul 2011, pratik wrote: > > >> Hi everyone, >> >> I am getting the following errors on testing PETSc on an SGI ALTIX cluster >> with sgi mpt-1.23.:(building and install was without any problems): >> >> /opt/intel/Compiler/11.1/038/bin/intel64/icc -o ex19.o -c -Wno-deprecated -g >> -I/home/pratikm/install/include -I/home/pratikm/install/include >> -I/usr/X11R6/include -I/usr/include -I/usr/lib64 >> -I/opt/sgi/mpt/mpt-1.23/include -D__INSDIR__=src/snes/examples/tutorials/ >> ex19.c >> /opt/intel/Compiler/11.1/038/bin/intel64/icc -Wno-deprecated -g -o ex19 >> ex19.o -Wl,-rpath,/home/pratikm/install/lib -L/home/pratikm/install/lib >> -lpetsc -L/usr/X11R6/lib64 -lX11 -Wl,-rpath,/usr/lib64 -L/usr/lib64 -lhdf5 >> -llapack -lblas -Wl,-rpath,/opt/sgi/mpt/mpt-1.23/lib >> -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -ldl >> -L/opt/intel/Compiler/11.1/038/lib/intel64 >> -L/usr/lib64/gcc/x86_64-suse-linux/4.1.2 -L/usr/x86_64-suse-linux/lib -limf >> -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lgfortran -lm -lm -ldl -limf >> -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -ldl >> /home/pratikm/install/lib/libpetsc.a(bvec2.o): In function >> `VecView_Seq_Binary': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/seq/bvec2.c:401: >> undefined reference to `mpi_sgi_status_ignore' >> /home/pratikm/install/lib/libpetsc.a(pdvec.o): In function >> `VecView_MPI_Binary': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/mpi/pdvec.c:459: >> undefined reference to `mpi_sgi_status_ignore' >> /home/pratikm/install/lib/libpetsc.a(gr2.o): In function `DAArrayMPIIO': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:478: undefined >> reference to `mpi_sgi_status_ignore' >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:480: undefined >> reference to `mpi_sgi_status_ignore' >> /home/pratikm/install/lib/libpetsc.a(vecio.o): In function `VecLoad_Binary': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/utils/vecio.c:275: >> undefined reference to `mpi_sgi_status_ignore' >> /home/pratikm/install/lib/libpetsc.a(mpiov.o): In function >> `MatGetSubMatrix_MPIAIJ_All': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:585: >> undefined reference to `mpi_sgi_inplace' >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:646: >> undefined reference to `mpi_sgi_inplace' >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:720: >> undefined reference to `mpi_sgi_inplace' >> /home/pratikm/install/lib/libpetsc.a(mpibaij.o): In function >> `MatGetSeqNonzerostructure_MPIBAIJ': >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2391: >> undefined reference to `mpi_sgi_inplace' >> /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2450: >> undefined reference to `mpi_sgi_inplace' >> make[3]: [ex19] Error 1 (ignored) >> /bin/rm -f ex19.o >> >> >> Can anyone please tell me what options i need to provide when configuring so >> that the program works correctly? >> here is the configure option that i used: >> >> ./config/configure.py --prefix=/home/pratikm/install >> -with-cc=/opt/intel/Compiler/11.1/038/bin/intel64/icc --CFLAGS=-Wno-deprecated >> -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 >> -xT -g -with-cxx=/opt/intel/Compiler/11.1/038/bin/intel64/icpc >> --CXXFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include >> -L/opt/sgi/mpt/mpt-1.23/lib -lmpi++ -lmpi -lsma >> --with-mpi-include=/opt/sgi/mpt/mpt-1.23/include >> --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so >> --with-hdf5-include=/usr/include --with-hdf5-lib=/usr/lib64/libhdf5.so >> >> >> Thanks in advance, >> Pratik >> >> > From bsmith at mcs.anl.gov Wed Jul 6 08:20:15 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 6 Jul 2011 08:20:15 -0500 Subject: [petsc-users] Question for time dependent problems In-Reply-To: References: <4E136824.2010405@nasa.gov> Message-ID: <152C8793-1F9F-4C61-A546-999140B733C5@mcs.anl.gov> On Jul 6, 2011, at 12:00 AM, Vijay S. Mahadevan wrote: > Barry, > > >> Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. > I found an earlier thread (http://lists.mcs.anl.gov/pipermail/petsc-users/2011-April/008566.html) where you said that it would not destroy the preallocation too. > > So is the behavior different in the dev version ? Just try it. > > Vijay > > On Jul 5, 2011 8:03 PM, "Barry Smith" wrote: > > > > On Jul 5, 2011, at 5:42 PM, Lisandro Dalcin wrote: > > > >> On 5 July 2011 17:35, Barry Smith wrote: > >>> > >>>> Beginning of a time step (1st implicit stage of RK scheme): > >>>> 1. Use MatZeroEntries on the preconditioner > >>> > >>> Do not call MatZeroEntries on a freshly created matrix (that destroys the preallocation pattern) so skip the MatZeroEntries the first time. > >>> > >> > >> What do you mean? The OP said that the nonzero structure was set at > >> problem set up. > > > > Perhaps I miss understood. I interpreted that to mean the matrix was provided with the correct preallocation information (and not the nonzero structure) but yes if the nonzero structure is already set then one can call MatZeroEntries() at that point. > > > > Perhaps we should "fix" MatZeroEntries()" to not screw up preallocation information! > > > > Barry > > > > > >> > >> -- > >> Lisandro Dalcin > >> --------------- > >> CIMEC (INTEC/CONICET-UNL) > >> Predio CONICET-Santa Fe > >> Colectora RN 168 Km 472, Paraje El Pozo > >> 3000 Santa Fe, Argentina > >> Tel: +54-342-4511594 (ext 1011) > >> Tel/Fax: +54-342-4511169 > > From bsmith at mcs.anl.gov Wed Jul 6 08:25:32 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 6 Jul 2011 08:25:32 -0500 Subject: [petsc-users] sgi mpt errors In-Reply-To: <4E13EA39.20205@gmail.com> References: <4E13EA39.20205@gmail.com> Message-ID: Pratik, As the PETSc documentation indicates if you have problems installing then please send configure.log and make.log to petsc-maint at mcs.anl.gov Thanks Barry On Jul 5, 2011, at 11:53 PM, pratik wrote: > Hi everyone, > > I am getting the following errors on testing PETSc on an SGI ALTIX cluster with sgi mpt-1.23.:(building and install was without any problems): > > /opt/intel/Compiler/11.1/038/bin/intel64/icc -o ex19.o -c -Wno-deprecated -g -I/home/pratikm/install/include -I/home/pratikm/install/include -I/usr/X11R6/include -I/usr/include -I/usr/lib64 -I/opt/sgi/mpt/mpt-1.23/include -D__INSDIR__=src/snes/examples/tutorials/ ex19.c > /opt/intel/Compiler/11.1/038/bin/intel64/icc -Wno-deprecated -g -o ex19 ex19.o -Wl,-rpath,/home/pratikm/install/lib -L/home/pratikm/install/lib -lpetsc -L/usr/X11R6/lib64 -lX11 -Wl,-rpath,/usr/lib64 -L/usr/lib64 -lhdf5 -llapack -lblas -Wl,-rpath,/opt/sgi/mpt/mpt-1.23/lib -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -ldl -L/opt/intel/Compiler/11.1/038/lib/intel64 -L/usr/lib64/gcc/x86_64-suse-linux/4.1.2 -L/usr/x86_64-suse-linux/lib -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lgfortran -lm -lm -ldl -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -ldl > /home/pratikm/install/lib/libpetsc.a(bvec2.o): In function `VecView_Seq_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/seq/bvec2.c:401: undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(pdvec.o): In function `VecView_MPI_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/impls/mpi/pdvec.c:459: undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(gr2.o): In function `DAArrayMPIIO': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:478: undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/dm/da/src/gr2.c:480: undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(vecio.o): In function `VecLoad_Binary': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/vec/vec/utils/vecio.c:275: undefined reference to `mpi_sgi_status_ignore' > /home/pratikm/install/lib/libpetsc.a(mpiov.o): In function `MatGetSubMatrix_MPIAIJ_All': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:585: undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:646: undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/aij/mpi/mpiov.c:720: undefined reference to `mpi_sgi_inplace' > /home/pratikm/install/lib/libpetsc.a(mpibaij.o): In function `MatGetSeqNonzerostructure_MPIBAIJ': > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2391: undefined reference to `mpi_sgi_inplace' > /home/pratikm/source/PETSc/petsc-3.1-p8/src/mat/impls/baij/mpi/mpibaij.c:2450: undefined reference to `mpi_sgi_inplace' > make[3]: [ex19] Error 1 (ignored) > /bin/rm -f ex19.o > > > Can anyone please tell me what options i need to provide when configuring so that the program works correctly? > here is the configure option that i used: > > ./config/configure.py --prefix=/home/pratikm/install -with-cc=/opt/intel/Compiler/11.1/038/bin/intel64/icc --CFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi -lsma -O3 -xT -g -with-cxx=/opt/intel/Compiler/11.1/038/bin/intel64/icpc --CXXFLAGS=-Wno-deprecated -I/opt/sgi/mpt/mpt-1.23/include -L/opt/sgi/mpt/mpt-1.23/lib -lmpi++ -lmpi -lsma --with-mpi-include=/opt/sgi/mpt/mpt-1.23/include --with-mpi-lib=/opt/sgi/mpt/mpt-1.23/lib/libmpi.so --with-hdf5-include=/usr/include --with-hdf5-lib=/usr/lib64/libhdf5.so > > > Thanks in advance, > Pratik From jedbrown at mcs.anl.gov Wed Jul 6 08:29:39 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 6 Jul 2011 08:29:39 -0500 Subject: [petsc-users] Question for time dependent problems In-Reply-To: <152C8793-1F9F-4C61-A546-999140B733C5@mcs.anl.gov> References: <4E136824.2010405@nasa.gov> <152C8793-1F9F-4C61-A546-999140B733C5@mcs.anl.gov> Message-ID: On Wed, Jul 6, 2011 at 08:20, Barry Smith wrote: > > >> Do not call MatZeroEntries on a freshly created matrix (that destroys > the preallocation pattern) so skip the MatZeroEntries the first time. > > I found an earlier thread ( > http://lists.mcs.anl.gov/pipermail/petsc-users/2011-April/008566.html) > where you said that it would not destroy the preallocation too. > > > > So is the behavior different in the dev version ? > > Just try it. PetscErrorCode MatZeroEntries_SeqAIJ(Mat A) { Mat_SeqAIJ *a = (Mat_SeqAIJ*)A->data; PetscErrorCode ierr; PetscFunctionBegin; ierr = PetscMemzero(a->a,(a->i[A->rmap->n])*sizeof(PetscScalar));CHKERRQ(ierr); PetscFunctionReturn(0); } This does not touch indices, therefore it does not destroy preallocation information. I can't think of a format where this operation would naturally destroy the information. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Wed Jul 6 09:16:43 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Wed, 6 Jul 2011 08:16:43 -0600 Subject: [petsc-users] Coupled multi-component problems with TS In-Reply-To: References: <7EF40573-CC56-4C8E-92C6-247EF9BF198D@mcmillan-mcgee.com> Message-ID: Hi Barry, Thank you very much for the help. I managed to figure it out with those new examples. Thanks again, Jonathan On 2011-06-29, at 8:08 PM, Barry Smith wrote: > > Jonathan, > > You need to switch to PETSc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html to access the DAE solvers that Jed Brown has developed. > > For that you use TSSetIFunction() and TSSetIJacobian() . > > Examples in src/ts/examples/tutorials > > ex14.c: ierr = TSSetIFunction(ts,THIFunction,thi);CHKERRQ(ierr); > ex15.c: ierr = TSSetIFunction(ts,FormIFunction,&user);CHKERRQ(ierr); > ex17.c: ierr = TSSetIFunction(ts,FormIFunction,&user);CHKERRQ(ierr); > ex18.c: ierr = TSSetIFunction(ts,FormResidual,&user);CHKERRQ(ierr); > ex8.c: ierr = TSSetIFunction(ts,problem->function,problem->data);CHKERRQ(ierr); > > all use this functionality. > > Good luck and feel free to send more questions once you are going, but I suspect it will be easy for you with this hint. > > > Barry > > On Jun 29, 2011, at 6:23 PM, Jonathan Backs wrote: > >> Hi, >> >> I am developing my first application with PETSc in an attempt to solve two coupled non-linear PDEs: (1) Poisson's equation for electric potential with a temperature-dependent electrical conductivity, and (2) the heat diffusion equation with a voltage-dependent thermal conductivity and a voltage-and-temperature-dependent source term (ohmic heating from the electric potential and the temperature-dependent electrical conductivity). >> >> So far I have solved the two problems separately without problems: Poisson's equation with a voltage-dependent electric conductivity (no timestepping), and the heat equation with a temperature-dependent thermal conductivity (timestepping, but no source term; the heat came from the boundary conditions in this case). However, I am not sure how to set up PETSc for solving these non-linear equations simultaneously. For example, when setting up a non-linear problem with time stepping, I understand I have to set the RHSFunction and RHSJacobian, assuming the left hand side provided by PETSc is the time-derivative of the independent variable; When setting up a non-linear problem without time stepping, I understand I have to set the Function assuming a zero left hand side (and then set the Jacobian according to the Function). >> >> I have tried to implement the coupled problem using a three-dimensional DA with two degrees of freedom (V and T). While PETSc seems to facilitate the use of multiple degrees of freedom in the same RHSFunction (and RHSJacobian), I cannot assign an RHSFunction for the Poisson's equation since the Poisson's equation does not include a time derivative. I have not been able to find any examples in the documentation or on this mailing list that illustrated this particular type of problem. >> >> Could anyone offer a hint as to the proper strategy for implementing this coupled system of PDEs? >> >> Thank you for your time, >> >> Jon > From ecoon at lanl.gov Wed Jul 6 09:41:10 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Wed, 06 Jul 2011 08:41:10 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> Message-ID: <1309963270.20331.10.camel@echo.lanl.gov> Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. Ethan On Mon, 2011-07-04 at 21:44 -0500, Barry Smith wrote: > What are the recommended visualization packages for use with PETSc (for example making movies of contour plots and isosurfaces) and what are the recommended data formats to use to save Vecs for visualization? > > Thanks > > Barry > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From adam1.byrd at gmail.com Wed Jul 6 09:53:44 2011 From: adam1.byrd at gmail.com (Adam Byrd) Date: Wed, 6 Jul 2011 10:53:44 -0400 Subject: [petsc-users] Matrix Construction Question In-Reply-To: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> References: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> Message-ID: I do not want the transpose, but that is what is coming out as the solution. Both my tests show the correct original matrix, and both yield the transpose of the solution verified against Mathematica and the website from which I pulled the 4x4 matrix in test.cpp. Could it be related to the factoring? On Tue, Jul 5, 2011 at 4:39 PM, Barry Smith wrote: > > MatSolve() and MatMatSolve() will not return the transpose of the > solution, they will return the solution. If you want the transpose solve you > can use MatSolveTranspose() for a single Vec or use MatMatSolve() for the > matrix case after transposing the original matrix if you want the transpose. > > Barry > > > On Jul 5, 2011, at 12:17 PM, Adam Byrd wrote: > > > I have working code that produces the correct answer now, thank you. One > (hopefully) final question, though. The solution is actually transposed. > What causes this? Presumably I can use MatMatSolveTranspose to get around > this, but there's no man page and I want to be sure of what will happen in > all cases. > > > > Respectfully, > > Adam > > > > On Thu, Jun 30, 2011 at 4:18 PM, Hong Zhang wrote: > > Add > > ierr = > MatGetOrdering(testMat,MATORDERING_ND,&isrow,&iscol);CHKERRQ(ierr); > > before the line > > ierr = MatFactorInfoInitialize(&luinfo);CHKERRQ(ierr); > > > > Somehow, your matrix is numerically singular. With > > MATORDERING_NATURAL > > I get > > [0]PETSC ERROR: Detected zero pivot in LU factorization > > > > even using MATDENSE matrix format, which calls lapack. > > With MATORDERING_ND, I get useless inverseMat. > > > > The modified code I used is attached. > > > > Hong > > > > On Thu, Jun 30, 2011 at 2:30 PM, Adam Byrd wrote: > > > I'm trying to work through what I need to do, again by practicing with > a > > > small scale random problem. The general order of events seems to be: > create > > > a matrix, fill it, assemble it, factor it, then one can use solvers > with it. > > > When I use MatLUFactor on my matrix before using it with a solver I get > this > > > error: > > > [0]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > [0]PETSC ERROR: Null argument, when expecting valid pointer! > > > [0]PETSC ERROR: Null Object: Parameter # 1! > > > [0]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > 13:37:48 > > > CDT 2011 > > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > [0]PETSC ERROR: See docs/index.html for manual pages. > > > [0]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: ./test on a osx-gnu-c named Macintosh-3.local by > adambyrd > > > Thu Jun 30 15:27:30 2011 > > > [0]PETSC ERROR: Libraries linked from > > > /Users/adambyrd/soft/petsc-3.1-p8/osx-gnu-cpp/lib > > > [0]PETSC ERROR: Configure run at Tue Jun 28 12:56:55 2011 > > > [0]PETSC ERROR: Configure options PETSC_ARCH=osx-gnu-cpp > --with-fc=gfortran > > > -download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex > > > --with-clanguage=c++ > > > [0]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: ISInvertPermutation() line 209 in > > > src/vec/is/interface/index.c > > > [0]PETSC ERROR: MatLUFactorSymbolic_SeqAIJ() line 306 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatLUFactorSymbolic() line 2534 in > > > src/mat/interface/matrix.c > > > [0]PETSC ERROR: MatLUFactor_SeqAIJ() line 945 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatLUFactor() line 2417 in src/mat/interface/matrix.c > > > [0]PETSC ERROR: main() line 62 in WDtest.cpp > > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0[unset]: > > > aborting job: > > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0 > > > I don't understand what I'm doing wrong. > > > Respectfully, > > > Adam > > > On Wed, Jun 29, 2011 at 1:26 AM, Matthew Knepley > wrote: > > >> > > >> On Tue, Jun 28, 2011 at 10:20 PM, Adam Byrd > wrote: > > >>> > > >>> Matt, > > >>> > > >>> Alright, that means I need to continue learning how to use > > >>> MatSetValues(). With my 6x6 example I tried filling it with four 3x3 > sub > > >>> matrices, but when I do that I get the error 'sum of local sizes 12 > does not > > >>> equal global size.' I had 4 processors each calling MatSetValues for > their > > >>> own 3x3. Graphically, I arranged the nodes 0 1 > > >>> > > >>> > 2 3 > > >>> where process 0 had global rows 0-2 and global columns 0-2; process 1 > had > > >>> 0-2, 3-5; process 2 had 3-5, 0-2; and process 3 had 3-5, 3-5. >From > the > > >>> documentation, I think this should be correct, but I'm not sure. > Also, which > > >>> format would you recommend for storing the matrix? > > >> > > >> 1) With any error, send the Entire error message. > > >> 2) PETSc matrices are divided by rows, not rows and columns, see the > > >> manual section. Rows & columns only makes sense for dense matrices > > >> 3) You can still set arbitrary blocks no matter how the matrix is > divided > > >> 4) The error means you tried to set both local and global dimensions, > and > > >> they do not add up correctly. Just set the global dimensions > > >> Matt > > >> > > >>> > > >>> Jack, > > >>> > > >>> I'm a summer intern just getting started with this project, so I > don't > > >>> know all the details yet (I can ask though). I know I need to find > the > > >>> Green's function which will involve the trace of the inverted > Hamiltonian, > > >>> as well as the rest of the matrix. I have inquired about avoiding the > > >>> inversion altogether, but my instructor doesn't believe there is a > way > > >>> around it. Once I've worked through the math I want to explore other > options > > >>> though. > > >>> > > >>> Respectfully, > > >>> Adam > > >>> > > >>> On Tue, Jun 28, 2011 at 6:08 PM, Matthew Knepley > > >>> wrote: > > >>>> > > >>>> On Tue, Jun 28, 2011 at 5:01 PM, Adam Byrd > wrote: > > >>>>> > > >>>>> Actually, it's quite sparse. In the 3600x3600 there are only just 4 > > >>>>> nonzero entries in each row. This means it's 99.9% empty. My > smaller 6x6 > > >>>>> example is dense, but it's only practice building and manipulating > matrices. > > >>>> > > >>>> Ah, then its easy. Just call MatSetValues() with each block. Then > use > > >>>> MUMPS to do a sparse direct solve. > > >>>> Matt > > >>>> > > >>>>> > > >>>>> Respectfully, > > >>>>> Adam > > >>>>> > > >>>>> On Tue, Jun 28, 2011 at 5:55 PM, Matthew Knepley < > knepley at gmail.com> > > >>>>> wrote: > > >>>>>> > > >>>>>> It sounds like you have a dense matrix (from your example). Is > this > > >>>>>> true? If so, you should use Elemental (on Google Code). > > >>>>>> Thanks, > > >>>>>> Matt > > >>>>>> > > >>>>>> On Tue, Jun 28, 2011 at 8:55 AM, Adam Byrd > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> Hi, > > >>>>>>> I'm rather new to PETSc and trying to work out the best way to > create > > >>>>>>> and fill a large sparse matrix distributed over many processors. > Currently, > > >>>>>>> my goal is to create a 3600x3600 matrix in units of 12x12 blocks > with > > >>>>>>> several blocks on any given node. I'd like to create the matrix > in such a > > >>>>>>> way that each node only holds the information in it's handful of > blocks and > > >>>>>>> not the entire matrix. Eventually, this matrix is to be inverted > (I know, > > >>>>>>> inversion should be avoided, but as this is a Hamiltonian matrix > from which > > >>>>>>> I need the Green's function, I'm unaware of a way to forgo > carrying out the > > >>>>>>> inversion). Additionally, the values will be changed slightly and > the matrix > > >>>>>>> will be repeatedly inverted. It's structure will remain the same. > In order > > >>>>>>> to learn how to do this is I am starting with a small 6x6 matrix > broken into > > >>>>>>> four 3x3 blocks and distributed one block per node. I've been > able to create > > >>>>>>> a local 3x3 matrix on each node, with it's own values, and with > the global > > >>>>>>> row/column IDs correctly set to [0, 1, 2] or [3, 4, 5] depending > on where > > >>>>>>> the block is in the matrix. My problem manifests when I try to > create the > > >>>>>>> larger matrix from the individual smaller ones. When the matrix > is > > >>>>>>> constructed I'm trying to use MatSetValues and having each node > pass in it's > > >>>>>>> 3x3 block. I end up with an error that the sum of local lengths > 12x12 does > > >>>>>>> not match the global length 6x6. It appears as though this is > from passing > > >>>>>>> in four 3x3s and the program interpreting that as a 12x12 instead > of as a > > >>>>>>> 6x6 with the blocks in a grid. > > >>>>>>> My question is then: is it possible to fill a matrix as a grid of > > >>>>>>> blocks, or can I only fill it in groups of rows or columns? Also, > am I > > >>>>>>> approaching this problem the correct way, or are there more > efficient ways > > >>>>>>> of building this matrix with the ultimate goal of inverting it? > > >>>>>>> I have included my copy of a modified example if it helps. I do > > >>>>>>> apologize if this is answered somewhere in the documentation, I > have been > > >>>>>>> unable to find a solution. > > >>>>>>> Respectfully, > > >>>>>>> Adam > > >>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> What most experimenters take for granted before they begin their > > >>>>>> experiments is infinitely more interesting than any results to > which their > > >>>>>> experiments lead. > > >>>>>> -- Norbert Wiener > > >>>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> What most experimenters take for granted before they begin their > > >>>> experiments is infinitely more interesting than any results to which > their > > >>>> experiments lead. > > >>>> -- Norbert Wiener > > >>> > > >> > > >> > > >> > > >> -- > > >> What most experimenters take for granted before they begin their > > >> experiments is infinitely more interesting than any results to which > their > > >> experiments lead. > > >> -- Norbert Wiener > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.cpp Type: text/x-c++src Size: 3119 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test2.cpp Type: text/x-c++src Size: 3754 bytes Desc: not available URL: From dalcinl at gmail.com Wed Jul 6 10:24:05 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 6 Jul 2011 12:24:05 -0300 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1309963270.20331.10.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> Message-ID: On 6 July 2011 11:41, Ethan Coon wrote: > Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. ?In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. ?For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. > Using petsc4py for reading PETSc binary formats is overkill. All this can be done with just NumPy. Should PETSc ship some Python code to read IS's, Vec's, Mat's in binary format? Can any of you suggest an appropriate location in the source tree? -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From ecoon at lanl.gov Wed Jul 6 11:56:55 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Wed, 06 Jul 2011 10:56:55 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> Message-ID: <1309971415.20331.15.camel@echo.lanl.gov> On Wed, 2011-07-06 at 12:24 -0300, Lisandro Dalcin wrote: > On 6 July 2011 11:41, Ethan Coon wrote: > > Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. > > > > Using petsc4py for reading PETSc binary formats is overkill. I guess if you don't have petsc4py installed, it may be overkill. But it's still a one-liner to do so if you do have petsc4py installed. I guess it does require knowing the size a priori, which should be accessible from the header, but otherwise it's not a big deal. > All this > can be done with just NumPy. Should PETSc ship some Python code to > read IS's, Vec's, Mat's in binary format? Can any of you suggest an > appropriate location in the source tree? Comparable code lives in bin/matlab... Ethan > > > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From bsmith at mcs.anl.gov Wed Jul 6 18:23:46 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 6 Jul 2011 18:23:46 -0500 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> Message-ID: Adam, I think you must be misunderstanding something. I have modified the program to multiply the original matrix by the solution and get back the identity so it is finding the inverse (also got rid of the c++ isms and changed it to match petsc-dev). Attached is the new program. Please explain why you think the true inverse is the transpose of what the code generates. Barry [see attached file: ex1.c] Matrix Object: 1 MPI processes type: seqaij row 0: (0, 10) (1, 4) (2, 6) (3, 9) (4, 9) (5, 5) (6, 1) (7, 7) (8, 2) (9, 2) (10, 10) (11, 4) row 1: (0, 0) (1, 2) (2, 5) (3, 9) (4, 3) (5, 3) (6, 2) (7, 9) (8, 6) (9, 1) (10, 9) (11, 5) row 2: (0, 5) (1, 9) (2, 5) (3, 5) (4, 9) (5, 10) (6, 8) (7, 0) (8, 9) (9, 3) (10, 6) (11, 2) row 3: (0, 9) (1, 7) (2, 10) (3, 7) (4, 6) (5, 3) (6, 4) (7, 6) (8, 2) (9, 1) (10, 1) (11, 2) row 4: (0, 3) (1, 3) (2, 2) (3, 7) (4, 3) (5, 7) (6, 7) (7, 3) (8, 9) (9, 9) (10, 1) (11, 7) row 5: (0, 7) (1, 3) (2, 4) (3, 6) (4, 4) (5, 1) (6, 7) (7, 8) (8, 8) (9, 4) (10, 6) (11, 2) row 6: (0, 7) (1, 6) (2, 9) (3, 0) (4, 10) (5, 9) (6, 2) (7, 2) (8, 7) (9, 6) (10, 7) (11, 4) row 7: (0, 6) (1, 0) (2, 9) (3, 4) (4, 8) (5, 5) (6, 3) (7, 1) (8, 8) (9, 9) (10, 10) (11, 3) row 8: (0, 7) (1, 5) (2, 8) (3, 1) (4, 0) (5, 2) (6, 7) (7, 9) (8, 1) (9, 8) (10, 1) (11, 0) row 9: (0, 9) (1, 9) (2, 9) (3, 0) (4, 8) (5, 10) (6, 1) (7, 10) (8, 7) (9, 10) (10, 8) (11, 10) row 10: (0, 0) (1, 6) (2, 2) (3, 1) (4, 1) (5, 8) (6, 4) (7, 8) (8, 6) (9, 2) (10, 3) (11, 9) row 11: (0, 4) (1, 7) (2, 2) (3, 0) (4, 7) (5, 6) (6, 0) (7, 2) (8, 5) (9, 6) (10, 7) (11, 1) Matrix Object: 1 MPI processes type: seqdense 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 Matrix Object: 1 MPI processes type: seqaij row 0: (0, 0.1) (1, 4) (2, 6) (3, 9) (4, 9) (5, 5) (6, 1) (7, 7) (8, 2) (9, 2) (10, 10) (11, 4) row 1: (0, 0) (1, 0.5) (2, 5) (3, 9) (4, 3) (5, 3) (6, 2) (7, 9) (8, 6) (9, 1) (10, 9) (11, 5) row 2: (0, 0.5) (1, 3.5) (2, -0.0645161) (3, -31) (4, -6) (5, -3) (6, 0.5) (7, -35) (8, -13) (9, -1.5) (10, -30.5) (11, -17.5) row 3: (0, 0.9) (1, 1.7) (2, 0.251613) (3, -0.116279) (4, -5.69032) (5, -5.84516) (6, -0.425806) (7, -6.79355) (8, -6.72903) (9, -2.12258) (10, -15.6258) (11, -5.69677) row 4: (0, 0.3) (1, 0.9) (2, 0.277419) (3, -0.55814) (4, -0.255658) (5, 0.369842) (6, 4.52363) (7, -1.28207) (8, 2.85071) (9, 6.73143) (10, -10.3601) (11, 2.97524) row 5: (0, 0.7) (1, 0.1) (2, 0.0451613) (3, -0.0232558) (4, 0.629267) (5, -0.329687) (6, 3.22094) (7, 4.42942) (8, 4.63675) (9, -1.71749) (10, 5.63329) (11, -2.51438) row 6: (0, 0.7) (1, 1.6) (2, 0.206452) (3, 1.66279) (4, -2.45445) (5, -3.93857) (6, 0.0444568) (7, 15.5209) (8, 35.1319) (9, 16.5966) (10, 14.638) (11, 3.68495) row 7: (0, 0.6) (1, -1.2) (2, -0.735484) (3, 1.55814) (4, -2.72363) (5, -4.45356) (6, 1.44469) (7, -0.0727551) (8, -7.41708) (9, -2.08795) (10, -7.56138) (11, -5.81272) row 8: (0, 0.7) (1, 1.1) (2, 0.109677) (3, 1.37209) (4, 0.289988) (5, -1.13475) (6, 0.309978) (7, -0.578186) (8, -0.141142) (9, -1.67585) (10, 9.37253) (11, -6.7832) row 9: (0, 0.9) (1, 2.7) (2, 0.63871) (3, 1.46512) (4, -1.01477) (5, -2.72166) (6, 0.371714) (7, -1.21452) (8, -0.0857132) (9, 0.347749) (10, 8.07184) (11, -0.411052) row 10: (0, 0) (1, 3) (2, 0.83871) (3, 4.13106e-16) (4, 0.758727) (5, -0.407335) (6, -0.201814) (7, -1.1833) (8, 0.431543) (9, -1.37252) (10, 0.0782676) (11, 1.62437) row 11: (0, 0.4) (1, 2.7) (2, 0.896774) (3, 0.0116279) (4, -0.190928) (5, 0.419064) (6, -0.299173) (7, -0.648141) (8, -0.570316) (9, 2.96665) (10, -1.34839) (11, 6.34192) Matrix Object: 1 MPI processes type: seqdense -9.6555301489395196e-01 5.2560254734804284e+00 7.1846440184462583e+00 -6.1331699956450372e+00 6.4983079560540347e-01 3.4596941129901471e-01 1.0107504375266285e+00 -6.8628221518606729e+00 7.5964800479384031e-01 9.9409527390937242e+00 -1.1538273684377408e+01 -8.7812172713296324e+00 2.0776013031269080e-02 -6.2847383863353556e-01 -6.6052840736344010e-01 8.0956641284817366e-01 -1.6685804755341449e-01 -6.9558502635303610e-02 -4.7043374819017370e-01 8.8565458307188294e-01 -1.4371117297291947e-01 -9.3869998563905455e-01 1.2914077666714618e+00 1.0574136681100246e+00 -2.3457711527316882e-01 9.1412301356649039e-01 1.1920152012252558e+00 -9.3204325120013032e-01 6.1684149677475715e-02 -5.5669152161757513e-03 1.5331009643936852e-01 -1.0495531120351829e+00 1.3707009393529770e-01 1.6438364660096838e+00 -1.8889315346051050e+00 -1.4754676224028302e+00 -2.7270521239062756e-01 1.5893219146430135e+00 2.0634754873336365e+00 -1.7209959749076202e+00 2.5337782971752582e-01 2.9994338843454111e-02 2.5534077638214847e-01 -1.9835560388079614e+00 2.1919306261832680e-01 2.8264140536952880e+00 -3.3585356810562166e+00 -2.4823956307658124e+00 1.7809631276633557e+00 -9.1417366043401938e+00 -1.2382493173582191e+01 1.0476797534295141e+01 -1.0702242259054311e+00 -4.4257528541486152e-01 -1.5514634845537427e+00 1.1648521003544433e+01 -1.3496709122035624e+00 -1.7047888183523145e+01 1.9724452609789598e+01 1.4992041216496093e+01 -9.8718243165411523e-01 5.8306164576852701e+00 7.6462462368614474e+00 -6.7409300641732255e+00 8.2470796373523481e-01 2.4183044407138940e-01 1.5018770736493510e+00 -7.5966352267475976e+00 9.5172226504897151e-01 1.0477013653074360e+01 -1.2424746738008650e+01 -9.4366200444961059e+00 1.0136311908130891e+00 -5.2691310897505188e+00 -6.8579746975031277e+00 5.9224590762975922e+00 -6.9910423550554868e-01 -2.6032159562560986e-01 -1.1422819623877292e+00 6.7605747527325546e+00 -7.2292745381698886e-01 -9.5958819114572318e+00 1.1268446709048680e+01 8.4067370921887932e+00 2.2091262060824907e-01 -8.3435384842300975e-01 -1.3941401244916276e+00 9.8039662735021815e-01 -3.8000873314291278e-02 8.9523733294413765e-03 1.4913622793099915e-01 9.8067426906640520e-01 -5.9946728984266576e-02 -1.8517463266334733e+00 1.9784368072815006e+00 1.5366714023840424e+00 -1.0121118595265313e+00 4.8899231637686178e+00 6.4363614363996273e+00 -5.5352594765178100e+00 6.3588032578740750e-01 3.6692073791888746e-01 1.0886795922616135e+00 -6.2890108659346788e+00 6.4878885440659029e-01 8.9338546663354137e+00 -1.0480724683245333e+01 -7.8880863422387675e+00 3.4828232494328459e-01 -1.8956049209699504e+00 -2.5686607685099623e+00 2.1723348669813345e+00 -1.8046993568519984e-01 -1.4627336724353771e-01 -4.3324999763575017e-01 2.4726324401858575e+00 -2.4110495922156525e-01 -3.4910576517078957e+00 4.0543680896562346e+00 3.1697497725756127e+00 -7.9100624200083347e-02 4.5467040831090605e-01 7.8405305146143733e-01 -5.8064262143845780e-01 -6.7175618016896713e-02 1.8327382703008988e-03 -1.7759194277182197e-01 -4.5740681002782790e-01 6.1885044052317102e-02 1.0071945338473738e+00 -1.0089198716799244e+00 -8.0628484740612849e-01 7.2503137134149309e-01 -4.0345450207039040e+00 -5.1688665129699460e+00 4.5837925160863282e+00 -5.6171429205502965e-01 -2.2648736352508647e-01 -1.1066305876267444e+00 5.2407359063046410e+00 -6.8000756820565422e-01 -7.0772451277993547e+00 8.5513959650299824e+00 6.3419246076729259e+00 anlextwls002-241:Downloads barrysmith$ ./ex1 Matrix Object: 1 MPI processes type: seqaij row 0: (0, 10) (1, 4) (2, 6) (3, 9) (4, 9) (5, 5) (6, 1) (7, 7) (8, 2) (9, 2) (10, 10) (11, 4) row 1: (0, 0) (1, 2) (2, 5) (3, 9) (4, 3) (5, 3) (6, 2) (7, 9) (8, 6) (9, 1) (10, 9) (11, 5) row 2: (0, 5) (1, 9) (2, 5) (3, 5) (4, 9) (5, 10) (6, 8) (7, 0) (8, 9) (9, 3) (10, 6) (11, 2) row 3: (0, 9) (1, 7) (2, 10) (3, 7) (4, 6) (5, 3) (6, 4) (7, 6) (8, 2) (9, 1) (10, 1) (11, 2) row 4: (0, 3) (1, 3) (2, 2) (3, 7) (4, 3) (5, 7) (6, 7) (7, 3) (8, 9) (9, 9) (10, 1) (11, 7) row 5: (0, 7) (1, 3) (2, 4) (3, 6) (4, 4) (5, 1) (6, 7) (7, 8) (8, 8) (9, 4) (10, 6) (11, 2) row 6: (0, 7) (1, 6) (2, 9) (3, 0) (4, 10) (5, 9) (6, 2) (7, 2) (8, 7) (9, 6) (10, 7) (11, 4) row 7: (0, 6) (1, 0) (2, 9) (3, 4) (4, 8) (5, 5) (6, 3) (7, 1) (8, 8) (9, 9) (10, 10) (11, 3) row 8: (0, 7) (1, 5) (2, 8) (3, 1) (4, 0) (5, 2) (6, 7) (7, 9) (8, 1) (9, 8) (10, 1) (11, 0) row 9: (0, 9) (1, 9) (2, 9) (3, 0) (4, 8) (5, 10) (6, 1) (7, 10) (8, 7) (9, 10) (10, 8) (11, 10) row 10: (0, 0) (1, 6) (2, 2) (3, 1) (4, 1) (5, 8) (6, 4) (7, 8) (8, 6) (9, 2) (10, 3) (11, 9) row 11: (0, 4) (1, 7) (2, 2) (3, 0) (4, 7) (5, 6) (6, 0) (7, 2) (8, 5) (9, 6) (10, 7) (11, 1) Matrix Object: 1 MPI processes type: seqdense 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00 Matrix Object: 1 MPI processes type: seqaij row 0: (0, 0.1) (1, 4) (2, 6) (3, 9) (4, 9) (5, 5) (6, 1) (7, 7) (8, 2) (9, 2) (10, 10) (11, 4) row 1: (0, 0) (1, 0.5) (2, 5) (3, 9) (4, 3) (5, 3) (6, 2) (7, 9) (8, 6) (9, 1) (10, 9) (11, 5) row 2: (0, 0.5) (1, 3.5) (2, -0.0645161) (3, -31) (4, -6) (5, -3) (6, 0.5) (7, -35) (8, -13) (9, -1.5) (10, -30.5) (11, -17.5) row 3: (0, 0.9) (1, 1.7) (2, 0.251613) (3, -0.116279) (4, -5.69032) (5, -5.84516) (6, -0.425806) (7, -6.79355) (8, -6.72903) (9, -2.12258) (10, -15.6258) (11, -5.69677) row 4: (0, 0.3) (1, 0.9) (2, 0.277419) (3, -0.55814) (4, -0.255658) (5, 0.369842) (6, 4.52363) (7, -1.28207) (8, 2.85071) (9, 6.73143) (10, -10.3601) (11, 2.97524) row 5: (0, 0.7) (1, 0.1) (2, 0.0451613) (3, -0.0232558) (4, 0.629267) (5, -0.329687) (6, 3.22094) (7, 4.42942) (8, 4.63675) (9, -1.71749) (10, 5.63329) (11, -2.51438) row 6: (0, 0.7) (1, 1.6) (2, 0.206452) (3, 1.66279) (4, -2.45445) (5, -3.93857) (6, 0.0444568) (7, 15.5209) (8, 35.1319) (9, 16.5966) (10, 14.638) (11, 3.68495) row 7: (0, 0.6) (1, -1.2) (2, -0.735484) (3, 1.55814) (4, -2.72363) (5, -4.45356) (6, 1.44469) (7, -0.0727551) (8, -7.41708) (9, -2.08795) (10, -7.56138) (11, -5.81272) row 8: (0, 0.7) (1, 1.1) (2, 0.109677) (3, 1.37209) (4, 0.289988) (5, -1.13475) (6, 0.309978) (7, -0.578186) (8, -0.141142) (9, -1.67585) (10, 9.37253) (11, -6.7832) row 9: (0, 0.9) (1, 2.7) (2, 0.63871) (3, 1.46512) (4, -1.01477) (5, -2.72166) (6, 0.371714) (7, -1.21452) (8, -0.0857132) (9, 0.347749) (10, 8.07184) (11, -0.411052) row 10: (0, 0) (1, 3) (2, 0.83871) (3, 4.13106e-16) (4, 0.758727) (5, -0.407335) (6, -0.201814) (7, -1.1833) (8, 0.431543) (9, -1.37252) (10, 0.0782676) (11, 1.62437) row 11: (0, 0.4) (1, 2.7) (2, 0.896774) (3, 0.0116279) (4, -0.190928) (5, 0.419064) (6, -0.299173) (7, -0.648141) (8, -0.570316) (9, 2.96665) (10, -1.34839) (11, 6.34192) Matrix Object: 1 MPI processes type: seqdense -9.6555301489395196e-01 5.2560254734804284e+00 7.1846440184462583e+00 -6.1331699956450372e+00 6.4983079560540347e-01 3.4596941129901471e-01 1.0107504375266285e+00 -6.8628221518606729e+00 7.5964800479384031e-01 9.9409527390937242e+00 -1.1538273684377408e+01 -8.7812172713296324e+00 2.0776013031269080e-02 -6.2847383863353556e-01 -6.6052840736344010e-01 8.0956641284817366e-01 -1.6685804755341449e-01 -6.9558502635303610e-02 -4.7043374819017370e-01 8.8565458307188294e-01 -1.4371117297291947e-01 -9.3869998563905455e-01 1.2914077666714618e+00 1.0574136681100246e+00 -2.3457711527316882e-01 9.1412301356649039e-01 1.1920152012252558e+00 -9.3204325120013032e-01 6.1684149677475715e-02 -5.5669152161757513e-03 1.5331009643936852e-01 -1.0495531120351829e+00 1.3707009393529770e-01 1.6438364660096838e+00 -1.8889315346051050e+00 -1.4754676224028302e+00 -2.7270521239062756e-01 1.5893219146430135e+00 2.0634754873336365e+00 -1.7209959749076202e+00 2.5337782971752582e-01 2.9994338843454111e-02 2.5534077638214847e-01 -1.9835560388079614e+00 2.1919306261832680e-01 2.8264140536952880e+00 -3.3585356810562166e+00 -2.4823956307658124e+00 1.7809631276633557e+00 -9.1417366043401938e+00 -1.2382493173582191e+01 1.0476797534295141e+01 -1.0702242259054311e+00 -4.4257528541486152e-01 -1.5514634845537427e+00 1.1648521003544433e+01 -1.3496709122035624e+00 -1.7047888183523145e+01 1.9724452609789598e+01 1.4992041216496093e+01 -9.8718243165411523e-01 5.8306164576852701e+00 7.6462462368614474e+00 -6.7409300641732255e+00 8.2470796373523481e-01 2.4183044407138940e-01 1.5018770736493510e+00 -7.5966352267475976e+00 9.5172226504897151e-01 1.0477013653074360e+01 -1.2424746738008650e+01 -9.4366200444961059e+00 1.0136311908130891e+00 -5.2691310897505188e+00 -6.8579746975031277e+00 5.9224590762975922e+00 -6.9910423550554868e-01 -2.6032159562560986e-01 -1.1422819623877292e+00 6.7605747527325546e+00 -7.2292745381698886e-01 -9.5958819114572318e+00 1.1268446709048680e+01 8.4067370921887932e+00 2.2091262060824907e-01 -8.3435384842300975e-01 -1.3941401244916276e+00 9.8039662735021815e-01 -3.8000873314291278e-02 8.9523733294413765e-03 1.4913622793099915e-01 9.8067426906640520e-01 -5.9946728984266576e-02 -1.8517463266334733e+00 1.9784368072815006e+00 1.5366714023840424e+00 -1.0121118595265313e+00 4.8899231637686178e+00 6.4363614363996273e+00 -5.5352594765178100e+00 6.3588032578740750e-01 3.6692073791888746e-01 1.0886795922616135e+00 -6.2890108659346788e+00 6.4878885440659029e-01 8.9338546663354137e+00 -1.0480724683245333e+01 -7.8880863422387675e+00 3.4828232494328459e-01 -1.8956049209699504e+00 -2.5686607685099623e+00 2.1723348669813345e+00 -1.8046993568519984e-01 -1.4627336724353771e-01 -4.3324999763575017e-01 2.4726324401858575e+00 -2.4110495922156525e-01 -3.4910576517078957e+00 4.0543680896562346e+00 3.1697497725756127e+00 -7.9100624200083347e-02 4.5467040831090605e-01 7.8405305146143733e-01 -5.8064262143845780e-01 -6.7175618016896713e-02 1.8327382703008988e-03 -1.7759194277182197e-01 -4.5740681002782790e-01 6.1885044052317102e-02 1.0071945338473738e+00 -1.0089198716799244e+00 -8.0628484740612849e-01 7.2503137134149309e-01 -4.0345450207039040e+00 -5.1688665129699460e+00 4.5837925160863282e+00 -5.6171429205502965e-01 -2.2648736352508647e-01 -1.1066305876267444e+00 5.2407359063046410e+00 -6.8000756820565422e-01 -7.0772451277993547e+00 8.5513959650299824e+00 6.3419246076729259e+00 Matrix Object: 1 MPI processes type: seqdense 9.9999999999999889e-01 7.1054273576010019e-15 -3.5527136788005009e-15 1.4210854715202004e-14 -1.3322676295501878e-15 2.2204460492503131e-16 8.8817841970012523e-16 1.0658141036401503e-14 -1.3322676295501878e-15 -1.4210854715202004e-14 2.1316282072803006e-14 -2.1316282072803006e-14 0.0000000000000000e+00 1.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 2.2204460492503131e-16 8.8817841970012523e-16 0.0000000000000000e+00 4.4408920985006262e-16 0.0000000000000000e+00 0.0000000000000000e+00 3.5527136788005009e-15 4.4408920985006262e-16 -1.9539925233402755e-14 1.0000000000000160e+00 -1.7763568394002505e-15 2.2204460492503131e-15 2.7755575615628914e-16 5.7731597280508140e-15 5.3290705182007514e-15 3.9968028886505635e-15 -3.0198066269804258e-14 -3.5527136788005009e-14 -2.8421709430404007e-14 1.5543122344752192e-15 1.0658141036401503e-14 3.7303493627405260e-14 1.0000000000000107e+00 2.4424906541753444e-15 1.4988010832439613e-15 6.6613381477509392e-15 -2.3092638912203256e-14 6.6613381477509392e-16 2.1316282072803006e-14 -3.5527136788005009e-15 -4.2632564145606011e-14 1.7763568394002505e-15 -7.1054273576010019e-15 1.4210854715202004e-14 1.4210854715202004e-14 9.9999999999999956e-01 4.4408920985006262e-16 8.8817841970012523e-16 1.4210854715202004e-14 0.0000000000000000e+00 -3.5527136788005009e-14 2.8421709430404007e-14 2.8421709430404007e-14 8.8817841970012523e-16 -1.2434497875801753e-14 -3.5527136788005009e-15 5.3290705182007514e-15 4.4408920985006262e-16 1.0000000000000009e+00 -4.4408920985006262e-16 7.1054273576010019e-15 -2.2204460492503131e-15 7.1054273576010019e-15 2.1316282072803006e-14 0.0000000000000000e+00 3.5527136788005009e-15 -2.1316282072803006e-14 -3.1974423109204508e-14 5.6843418860808015e-14 -1.3322676295501878e-15 -1.2212453270876722e-15 1.0000000000000062e+00 -2.8421709430404007e-14 -2.6645352591003757e-15 1.0658141036401503e-14 -2.8421709430404007e-14 0.0000000000000000e+00 1.7763568394002505e-15 8.8817841970012523e-15 -1.7763568394002505e-15 5.6843418860808015e-14 -8.8817841970012523e-16 -1.1102230246251565e-16 9.7699626167013776e-15 9.9999999999995204e-01 -2.2204460492503131e-15 3.9079850466805510e-14 -4.2632564145606011e-14 -1.0658141036401503e-14 2.2204460492503131e-16 -6.2727600891321345e-15 6.5503158452884236e-15 2.9976021664879227e-15 3.5943470422239443e-15 1.9485281443909486e-15 7.5772721430666934e-15 -3.6637359812630166e-14 1.0000000000000007e+00 1.2656542480726785e-14 -1.6875389974302379e-14 -2.4313884239290928e-14 -3.5527136788005009e-15 2.1316282072803006e-14 1.4210854715202004e-14 -2.1316282072803006e-14 3.5527136788005009e-15 1.3322676295501878e-15 1.4210854715202004e-14 -3.5527136788005009e-14 3.5527136788005009e-15 1.0000000000000000e+00 -5.6843418860808015e-14 -6.3948846218409017e-14 1.7763568394002505e-15 -2.1316282072803006e-14 -7.1054273576010019e-15 0.0000000000000000e+00 8.8817841970012523e-16 0.0000000000000000e+00 0.0000000000000000e+00 7.1054273576010019e-15 2.6645352591003757e-15 -3.5527136788005009e-14 1.0000000000000000e+00 7.1054273576010019e-15 -2.4424906541753444e-15 1.2434497875801753e-14 4.5297099404706387e-14 -4.6185277824406512e-14 4.1078251911130792e-15 1.5265566588595902e-15 8.4376949871511897e-15 -2.5757174171303632e-14 5.2180482157382357e-15 3.9079850466805510e-14 -7.8159700933611020e-14 9.9999999999994316e-01 On Jul 6, 2011, at 9:53 AM, Adam Byrd wrote: > I do not want the transpose, but that is what is coming out as the solution. Both my tests show the correct original matrix, and both yield the transpose of the solution verified against Mathematica and the website from which I pulled the 4x4 matrix in test.cpp. Could it be related to the factoring? > > On Tue, Jul 5, 2011 at 4:39 PM, Barry Smith wrote: > > MatSolve() and MatMatSolve() will not return the transpose of the solution, they will return the solution. If you want the transpose solve you can use MatSolveTranspose() for a single Vec or use MatMatSolve() for the matrix case after transposing the original matrix if you want the transpose. > > Barry > > > On Jul 5, 2011, at 12:17 PM, Adam Byrd wrote: > > > I have working code that produces the correct answer now, thank you. One (hopefully) final question, though. The solution is actually transposed. What causes this? Presumably I can use MatMatSolveTranspose to get around this, but there's no man page and I want to be sure of what will happen in all cases. > > > > Respectfully, > > Adam > > > > On Thu, Jun 30, 2011 at 4:18 PM, Hong Zhang wrote: > > Add > > ierr = MatGetOrdering(testMat,MATORDERING_ND,&isrow,&iscol);CHKERRQ(ierr); > > before the line > > ierr = MatFactorInfoInitialize(&luinfo);CHKERRQ(ierr); > > > > Somehow, your matrix is numerically singular. With > > MATORDERING_NATURAL > > I get > > [0]PETSC ERROR: Detected zero pivot in LU factorization > > > > even using MATDENSE matrix format, which calls lapack. > > With MATORDERING_ND, I get useless inverseMat. > > > > The modified code I used is attached. > > > > Hong > > > > On Thu, Jun 30, 2011 at 2:30 PM, Adam Byrd wrote: > > > I'm trying to work through what I need to do, again by practicing with a > > > small scale random problem. The general order of events seems to be: create > > > a matrix, fill it, assemble it, factor it, then one can use solvers with it. > > > When I use MatLUFactor on my matrix before using it with a solver I get this > > > error: > > > [0]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > [0]PETSC ERROR: Null argument, when expecting valid pointer! > > > [0]PETSC ERROR: Null Object: Parameter # 1! > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 > > > CDT 2011 > > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > [0]PETSC ERROR: See docs/index.html for manual pages. > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: ./test on a osx-gnu-c named Macintosh-3.local by adambyrd > > > Thu Jun 30 15:27:30 2011 > > > [0]PETSC ERROR: Libraries linked from > > > /Users/adambyrd/soft/petsc-3.1-p8/osx-gnu-cpp/lib > > > [0]PETSC ERROR: Configure run at Tue Jun 28 12:56:55 2011 > > > [0]PETSC ERROR: Configure options PETSC_ARCH=osx-gnu-cpp --with-fc=gfortran > > > -download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex > > > --with-clanguage=c++ > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: ISInvertPermutation() line 209 in > > > src/vec/is/interface/index.c > > > [0]PETSC ERROR: MatLUFactorSymbolic_SeqAIJ() line 306 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatLUFactorSymbolic() line 2534 in > > > src/mat/interface/matrix.c > > > [0]PETSC ERROR: MatLUFactor_SeqAIJ() line 945 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatLUFactor() line 2417 in src/mat/interface/matrix.c > > > [0]PETSC ERROR: main() line 62 in WDtest.cpp > > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0[unset]: > > > aborting job: > > > application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0 > > > I don't understand what I'm doing wrong. > > > Respectfully, > > > Adam > > > On Wed, Jun 29, 2011 at 1:26 AM, Matthew Knepley wrote: > > >> > > >> On Tue, Jun 28, 2011 at 10:20 PM, Adam Byrd wrote: > > >>> > > >>> Matt, > > >>> > > >>> Alright, that means I need to continue learning how to use > > >>> MatSetValues(). With my 6x6 example I tried filling it with four 3x3 sub > > >>> matrices, but when I do that I get the error 'sum of local sizes 12 does not > > >>> equal global size.' I had 4 processors each calling MatSetValues for their > > >>> own 3x3. Graphically, I arranged the nodes 0 1 > > >>> > > >>> 2 3 > > >>> where process 0 had global rows 0-2 and global columns 0-2; process 1 had > > >>> 0-2, 3-5; process 2 had 3-5, 0-2; and process 3 had 3-5, 3-5. >From the > > >>> documentation, I think this should be correct, but I'm not sure. Also, which > > >>> format would you recommend for storing the matrix? > > >> > > >> 1) With any error, send the Entire error message. > > >> 2) PETSc matrices are divided by rows, not rows and columns, see the > > >> manual section. Rows & columns only makes sense for dense matrices > > >> 3) You can still set arbitrary blocks no matter how the matrix is divided > > >> 4) The error means you tried to set both local and global dimensions, and > > >> they do not add up correctly. Just set the global dimensions > > >> Matt > > >> > > >>> > > >>> Jack, > > >>> > > >>> I'm a summer intern just getting started with this project, so I don't > > >>> know all the details yet (I can ask though). I know I need to find the > > >>> Green's function which will involve the trace of the inverted Hamiltonian, > > >>> as well as the rest of the matrix. I have inquired about avoiding the > > >>> inversion altogether, but my instructor doesn't believe there is a way > > >>> around it. Once I've worked through the math I want to explore other options > > >>> though. > > >>> > > >>> Respectfully, > > >>> Adam > > >>> > > >>> On Tue, Jun 28, 2011 at 6:08 PM, Matthew Knepley > > >>> wrote: > > >>>> > > >>>> On Tue, Jun 28, 2011 at 5:01 PM, Adam Byrd wrote: > > >>>>> > > >>>>> Actually, it's quite sparse. In the 3600x3600 there are only just 4 > > >>>>> nonzero entries in each row. This means it's 99.9% empty. My smaller 6x6 > > >>>>> example is dense, but it's only practice building and manipulating matrices. > > >>>> > > >>>> Ah, then its easy. Just call MatSetValues() with each block. Then use > > >>>> MUMPS to do a sparse direct solve. > > >>>> Matt > > >>>> > > >>>>> > > >>>>> Respectfully, > > >>>>> Adam > > >>>>> > > >>>>> On Tue, Jun 28, 2011 at 5:55 PM, Matthew Knepley > > >>>>> wrote: > > >>>>>> > > >>>>>> It sounds like you have a dense matrix (from your example). Is this > > >>>>>> true? If so, you should use Elemental (on Google Code). > > >>>>>> Thanks, > > >>>>>> Matt > > >>>>>> > > >>>>>> On Tue, Jun 28, 2011 at 8:55 AM, Adam Byrd > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> Hi, > > >>>>>>> I'm rather new to PETSc and trying to work out the best way to create > > >>>>>>> and fill a large sparse matrix distributed over many processors. Currently, > > >>>>>>> my goal is to create a 3600x3600 matrix in units of 12x12 blocks with > > >>>>>>> several blocks on any given node. I'd like to create the matrix in such a > > >>>>>>> way that each node only holds the information in it's handful of blocks and > > >>>>>>> not the entire matrix. Eventually, this matrix is to be inverted (I know, > > >>>>>>> inversion should be avoided, but as this is a Hamiltonian matrix from which > > >>>>>>> I need the Green's function, I'm unaware of a way to forgo carrying out the > > >>>>>>> inversion). Additionally, the values will be changed slightly and the matrix > > >>>>>>> will be repeatedly inverted. It's structure will remain the same. In order > > >>>>>>> to learn how to do this is I am starting with a small 6x6 matrix broken into > > >>>>>>> four 3x3 blocks and distributed one block per node. I've been able to create > > >>>>>>> a local 3x3 matrix on each node, with it's own values, and with the global > > >>>>>>> row/column IDs correctly set to [0, 1, 2] or [3, 4, 5] depending on where > > >>>>>>> the block is in the matrix. My problem manifests when I try to create the > > >>>>>>> larger matrix from the individual smaller ones. When the matrix is > > >>>>>>> constructed I'm trying to use MatSetValues and having each node pass in it's > > >>>>>>> 3x3 block. I end up with an error that the sum of local lengths 12x12 does > > >>>>>>> not match the global length 6x6. It appears as though this is from passing > > >>>>>>> in four 3x3s and the program interpreting that as a 12x12 instead of as a > > >>>>>>> 6x6 with the blocks in a grid. > > >>>>>>> My question is then: is it possible to fill a matrix as a grid of > > >>>>>>> blocks, or can I only fill it in groups of rows or columns? Also, am I > > >>>>>>> approaching this problem the correct way, or are there more efficient ways > > >>>>>>> of building this matrix with the ultimate goal of inverting it? > > >>>>>>> I have included my copy of a modified example if it helps. I do > > >>>>>>> apologize if this is answered somewhere in the documentation, I have been > > >>>>>>> unable to find a solution. > > >>>>>>> Respectfully, > > >>>>>>> Adam > > >>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> What most experimenters take for granted before they begin their > > >>>>>> experiments is infinitely more interesting than any results to which their > > >>>>>> experiments lead. > > >>>>>> -- Norbert Wiener > > >>>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> What most experimenters take for granted before they begin their > > >>>> experiments is infinitely more interesting than any results to which their > > >>>> experiments lead. > > >>>> -- Norbert Wiener > > >>> > > >> > > >> > > >> > > >> -- > > >> What most experimenters take for granted before they begin their > > >> experiments is infinitely more interesting than any results to which their > > >> experiments lead. > > >> -- Norbert Wiener > > > > > > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: ex1.c Type: application/octet-stream Size: 4036 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Wed Jul 6 23:20:19 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 6 Jul 2011 23:20:19 -0500 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> Message-ID: On Wed, Jul 6, 2011 at 09:53, Adam Byrd wrote: > I do not want the transpose, but that is what is coming out as the > solution. Both my tests show the correct original matrix, and both yield the > transpose of the solution verified against Mathematica and the website from > which I pulled the 4x4 matrix in test.cpp. Could it be related to the > factoring? The problem is that you are trying to solve with an AIJ matrix instead of a DENSE matrix. MatMatSolve() calls MatGetArray(). Since this is serial and you preallocated enough for a dense matrix, it goes through with no memory errors. The storage format for MATAIJ (that happens to be dense) and MATDENSE is transposed, so you see the transpose. If you make the solution matrix dense, then it will work correctly. --- a/test.cpp +++ b/test.cpp @@ -48,7 +48,9 @@ int main(int argc,char **args) ierr = MatAssemblyEnd(identityMat, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); //Setup inverseMat - ierr = MatDuplicate(testMat, MAT_DO_NOT_COPY_VALUES, &inverseMat);CHKERRQ(ierr); + ierr = MatCreate(PETSC_COMM_WORLD, &inverseMat);CHKERRQ(ierr); + ierr = MatSetSizes(inverseMat, PETSC_DECIDE, PETSC_DECIDE, rows, columns);CHKERRQ(ierr); + ierr = MatSetType(inverseMat, MATDENSE);CHKERRQ(ierr); ierr = MatAssemblyBegin(inverseMat, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(inverseMat, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); PETSc should be more careful about matching types here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andrew.Barker at Colorado.EDU Thu Jul 7 09:21:07 2011 From: Andrew.Barker at Colorado.EDU (Andrew T Barker) Date: Thu, 7 Jul 2011 08:21:07 -0600 (MDT) Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> Message-ID: <20110707082107.AJC19221@batman.int.colorado.edu> > Should PETSc ship some Python code to > read IS's, Vec's, Mat's in binary format? Yes. Please. I have played with matplotlib some to view Petsc output (vtk), with mixed results. I wonder if Ethan would be willing to share an example or two? Best, Andrew From marco.cisternino at polito.it Thu Jul 7 11:55:49 2011 From: marco.cisternino at polito.it (Marco Cisternino) Date: Thu, 07 Jul 2011 18:55:49 +0200 Subject: [petsc-users] scattering and communications Message-ID: <4E15E515.6020806@polito.it> Hi, I would like to understand better how VecScatterBegin and VecScatterEnd work. I solve an interface elliptic problem on a Cartesian grid introducing extra unknowns (intersections between the interface and the grid axes). Therefore, my linear system is augmented with extra condition on these new unknowns. I use a DA to manage the grid, but I have to use MatCreateMPIAIJ to build the matrix because of the nature of the problem. MatCreateMPIAIJ(proc->cart_comm,proc->intersections[proc->rank],proc->intersections[proc->rank],g_rows,g_cols,21,PETSC_NULL,21,PETSC_NULL,&fsolv->AA); VecCreateMPI(proc->cart_comm,proc->intersections[proc->rank],PETSC_DECIDE,&fsolv->extended_P); VecCreateMPI(proc->cart_comm,proc->intersections[proc->rank],PETSC_DECIDE,&fsolv->extended_RHS); where proc->intersections[proc->rank] is the total number of unknowns for each processor in its sub-domain (grid points + intersections). g_rows=g_cols is the total number of unknowns in the entire computational domain (grid points + intersections). cart_comm is a Cartesian communicator. The arrangement of the unknowns is such that every processor has the rows of the matrix and of extended_P(the solution) relative to the actual unknowns in its sub-domain. I solve the system and then I call VecScatterBegin and VecScatterEnd: ierr=VecScatterCreate(fsolv->extended_P,scatter->from,vec->GP,scatter->to,&scatter->scatt); ierr=VecScatterBegin(scatter->scatt,fsolv->extended_P,vec->GP,INSERT_VALUES,SCATTER_FORWARD); ierr=VecScatterEnd(scatter->scatt,fsolv->extended_P,vec->GP,INSERT_VALUES,SCATTER_FORWARD); in order to get in GP (made using DACreateGlobalVector(grid->da,&vec->GP); ) only the solution on the grid points. It works, I mean I can get the right solution in GP, but the scattering doesn't scale at all! I would expect no communications during the scattering, doing what I do, but -log_summary shows me a number of MPI message growing with the number of processors. Every portion of GP contains only the grid points of a processor, while every portion of extended_P contains the same grid points plus the intersections in the relative sub-domain. which is the need for the communications doing such a scattering? I don't know if I was clear enough. Please, ask me what you need to understand my problem. Thanks a lot. Best regards, Marco -- Marco Cisternino PhD Student Politecnico di Torino Mobile:+393281189696 Email:marco.cisternino at polito.it From adam1.byrd at gmail.com Thu Jul 7 12:02:51 2011 From: adam1.byrd at gmail.com (Adam Byrd) Date: Thu, 7 Jul 2011 13:02:51 -0400 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> Message-ID: Changing the matrix types worked. When solving AX=B, X and B have to be dense and A can be a sparse format? I'd like to avoid storing the identity as a dense format, if possible. By the way, the man page for MATAIJ says it's a sparse format http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MATAIJ.html . I must ask, is there somewhere in the documentation I should be looking to figure these sorts of questions out? I feel like a pest, but I'm not sure where else to look for answers. On Thu, Jul 7, 2011 at 12:20 AM, Jed Brown wrote: > On Wed, Jul 6, 2011 at 09:53, Adam Byrd wrote: > >> I do not want the transpose, but that is what is coming out as the >> solution. Both my tests show the correct original matrix, and both yield the >> transpose of the solution verified against Mathematica and the website from >> which I pulled the 4x4 matrix in test.cpp. Could it be related to the >> factoring? > > > The problem is that you are trying to solve with an AIJ matrix instead of a > DENSE matrix. MatMatSolve() calls MatGetArray(). Since this is serial and > you preallocated enough for a dense matrix, it goes through with no memory > errors. The storage format for MATAIJ (that happens to be dense) and > MATDENSE is transposed, so you see the transpose. If you make the solution > matrix dense, then it will work correctly. > > > --- a/test.cpp > +++ b/test.cpp > @@ -48,7 +48,9 @@ int main(int argc,char **args) > ierr = MatAssemblyEnd(identityMat, > MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > > //Setup inverseMat > - ierr = MatDuplicate(testMat, MAT_DO_NOT_COPY_VALUES, > &inverseMat);CHKERRQ(ierr); > + ierr = MatCreate(PETSC_COMM_WORLD, &inverseMat);CHKERRQ(ierr); > + ierr = MatSetSizes(inverseMat, PETSC_DECIDE, PETSC_DECIDE, rows, > columns);CHKERRQ(ierr); > + ierr = MatSetType(inverseMat, MATDENSE);CHKERRQ(ierr); > ierr = MatAssemblyBegin(inverseMat, > MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > ierr = MatAssemblyEnd(inverseMat, > MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > > > > PETSc should be more careful about matching types here. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecoon at lanl.gov Thu Jul 7 12:12:40 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Thu, 07 Jul 2011 11:12:40 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <20110707082107.AJC19221@batman.int.colorado.edu> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <20110707082107.AJC19221@batman.int.colorado.edu> Message-ID: <1310058760.24206.54.camel@echo.lanl.gov> Hmm, I've been tricked apparently. What I thought was PETSc writing out VTK was actually PFLOTRAN writing out VTK from PETSc data structures. On a day-to-day basis I tend to use HDF5, and I've checked that that does, in fact, work with only PETSc, and only a few lines of PETSc. Follow/run example: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/dm/da/examples/tutorials/ex9.c.html Then load that in python with h5py and view with matplotlib: In [1]: import h5py In [2]: ex9 = h5py.File('hdf5output') In [3]: ex9 Out[3]: In [4]: ex9.keys() Out[4]: ['Vec_0x84000000_0'] In [5]: vec = ex9[ex9.keys()[0]] In [6]: vec Out[6]: In [7]: vec.shape # vec was created from a 2D DA, size (90,100), with 2 dofs Out[7]: (90, 100, 2) In [8]: from matplotlib import pyplot as plt In [9]: plt.imshow(vec[:,:,0].transpose(), origin='lower') Out[9]: In [10]: plt.colorbar() Out[10]: In [11]: plt.show() The downside of this option is that, in order for hdf5 files to work in anything but python (which is smart enough to use introspection on the file itself) you need a special viewer (the VisIt people call this a "flavor" of HDF5), see http://visitusers.org/index.php?title=VisIt_and_HDF5 These work on both VisIt and Paraview, but I've never tried to write one myself, so I don't know how ugly they are. They basically specify the layout of the hdf5 file. Note that if you go with the HDF5 option, you'll likely want to move to petsc-dev, which includes more HDF5 functionality, such as groups. Ethan On Thu, 2011-07-07 at 08:21 -0600, Andrew T Barker wrote: > > Should PETSc ship some Python code to > > read IS's, Vec's, Mat's in binary format? > > Yes. Please. > > I have played with matplotlib some to view Petsc output (vtk), with mixed results. I wonder if Ethan would be willing to share an example or two? > > Best, > > Andrew -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From knepley at gmail.com Thu Jul 7 13:09:34 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 7 Jul 2011 18:09:34 +0000 Subject: [petsc-users] Matrix Construction Question In-Reply-To: References: <751C31A8-3427-4B37-9FBF-5D93AD28A05B@mcs.anl.gov> Message-ID: On Thu, Jul 7, 2011 at 5:02 PM, Adam Byrd wrote: > Changing the matrix types worked. When solving AX=B, X and B have to be > dense and A can be a sparse format? I'd like to avoid storing the identity > as a dense format, if possible. By the way, the man page for MATAIJ says > it's a sparse format > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MATAIJ.html > . Yes, AIJ is sparse. > I must ask, is there somewhere in the documentation I should be looking to > figure these sorts of questions out? I feel like a pest, but I'm not sure > where else to look for answers. > Which questions? If you mean about the MatMatSolve, it says the arguments must be dense on the webpage: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatMatSolve.html There is also a manual, FAQ, and hundreds of hyperlinked examples. Matt > On Thu, Jul 7, 2011 at 12:20 AM, Jed Brown wrote: > >> On Wed, Jul 6, 2011 at 09:53, Adam Byrd wrote: >> >>> I do not want the transpose, but that is what is coming out as the >>> solution. Both my tests show the correct original matrix, and both yield the >>> transpose of the solution verified against Mathematica and the website from >>> which I pulled the 4x4 matrix in test.cpp. Could it be related to the >>> factoring? >> >> >> The problem is that you are trying to solve with an AIJ matrix instead of >> a DENSE matrix. MatMatSolve() calls MatGetArray(). Since this is serial and >> you preallocated enough for a dense matrix, it goes through with no memory >> errors. The storage format for MATAIJ (that happens to be dense) and >> MATDENSE is transposed, so you see the transpose. If you make the solution >> matrix dense, then it will work correctly. >> >> >> --- a/test.cpp >> +++ b/test.cpp >> @@ -48,7 +48,9 @@ int main(int argc,char **args) >> ierr = MatAssemblyEnd(identityMat, >> MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); >> >> //Setup inverseMat >> - ierr = MatDuplicate(testMat, MAT_DO_NOT_COPY_VALUES, >> &inverseMat);CHKERRQ(ierr); >> + ierr = MatCreate(PETSC_COMM_WORLD, &inverseMat);CHKERRQ(ierr); >> + ierr = MatSetSizes(inverseMat, PETSC_DECIDE, PETSC_DECIDE, rows, >> columns);CHKERRQ(ierr); >> + ierr = MatSetType(inverseMat, MATDENSE);CHKERRQ(ierr); >> ierr = MatAssemblyBegin(inverseMat, >> MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); >> ierr = MatAssemblyEnd(inverseMat, >> MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); >> >> >> >> PETSc should be more careful about matching types here. >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Jul 7 13:14:18 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 7 Jul 2011 18:14:18 +0000 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1310058760.24206.54.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <20110707082107.AJC19221@batman.int.colorado.edu> <1310058760.24206.54.camel@echo.lanl.gov> Message-ID: On Thu, Jul 7, 2011 at 5:12 PM, Ethan Coon wrote: > Hmm, I've been tricked apparently. What I thought was PETSc writing out > VTK was actually PFLOTRAN writing out VTK from PETSc data structures. > PETSc will write out VTK if you a) set the PETSC_VIEWER_ASCII_VTK format b) view the DA first and then Vecs Matt > On a day-to-day basis I tend to use HDF5, and I've checked that that > does, in fact, work with only PETSc, and only a few lines of PETSc. > Follow/run example: > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/dm/da/examples/tutorials/ex9.c.html > > Then load that in python with h5py and view with matplotlib: > > In [1]: import h5py > > In [2]: ex9 = h5py.File('hdf5output') > > In [3]: ex9 > Out[3]: > > In [4]: ex9.keys() > Out[4]: ['Vec_0x84000000_0'] > > In [5]: vec = ex9[ex9.keys()[0]] > > In [6]: vec > Out[6]: " > > In [7]: vec.shape # vec was created from a 2D DA, size (90,100), with 2 > dofs > Out[7]: (90, 100, 2) > > In [8]: from matplotlib import pyplot as plt > > In [9]: plt.imshow(vec[:,:,0].transpose(), origin='lower') > Out[9]: > > In [10]: plt.colorbar() > Out[10]: > > In [11]: plt.show() > > > The downside of this option is that, in order for hdf5 files to work in > anything but python (which is smart enough to use introspection on the > file itself) you need a special viewer (the VisIt people call this a > "flavor" of HDF5), see > > http://visitusers.org/index.php?title=VisIt_and_HDF5 > > These work on both VisIt and Paraview, but I've never tried to write one > myself, so I don't know how ugly they are. They basically specify the > layout of the hdf5 file. > > Note that if you go with the HDF5 option, you'll likely want to move to > petsc-dev, which includes more HDF5 functionality, such as groups. > > Ethan > > > On Thu, 2011-07-07 at 08:21 -0600, Andrew T Barker wrote: > > > Should PETSc ship some Python code to > > > read IS's, Vec's, Mat's in binary format? > > > > Yes. Please. > > > > I have played with matplotlib some to view Petsc output (vtk), with mixed > results. I wonder if Ethan would be willing to share an example or two? > > > > Best, > > > > Andrew > > -- > ------------------------------------ > Ethan Coon > Post-Doctoral Researcher > Applied Mathematics - T-5 > Los Alamos National Laboratory > 505-665-8289 > > http://www.ldeo.columbia.edu/~ecoon/ > ------------------------------------ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From baagaard at usgs.gov Thu Jul 7 13:31:11 2011 From: baagaard at usgs.gov (Brad Aagaard) Date: Thu, 07 Jul 2011 11:31:11 -0700 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1310058760.24206.54.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <20110707082107.AJC19221@batman.int.colorado.edu> <1310058760.24206.54.camel@echo.lanl.gov> Message-ID: <4E15FB6F.4000704@usgs.gov> Ethan- I think writing an Xdmf file that points to the datasets in an HDF5 file is probably easier than creating a viewer. For PyLith we write the HDF5 file and a corresponding Xdmf file. This allows us to simply open the Xdmf file in ParaView; I think VisIt supports this as well. Brad On 07/07/2011 10:12 AM, Ethan Coon wrote: > The downside of this option is that, in order for hdf5 files to work in > anything but python (which is smart enough to use introspection on the > file itself) you need a special viewer (the VisIt people call this a > "flavor" of HDF5), see > > http://visitusers.org/index.php?title=VisIt_and_HDF5 > > These work on both VisIt and Paraview, but I've never tried to write one > myself, so I don't know how ugly they are. They basically specify the > layout of the hdf5 file. > > Note that if you go with the HDF5 option, you'll likely want to move to > petsc-dev, which includes more HDF5 functionality, such as groups. From bourdin at lsu.edu Thu Jul 7 15:01:14 2011 From: bourdin at lsu.edu (Blaise A Bourdin) Date: Thu, 7 Jul 2011 22:01:14 +0200 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <4E15FB6F.4000704@usgs.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <20110707082107.AJC19221@batman.int.colorado.edu> <1310058760.24206.54.camel@echo.lanl.gov> <4E15FB6F.4000704@usgs.gov> Message-ID: <15D137AD-7861-4C61-8876-AE816C45B9DF@lsu.edu> Brad, Do you have examples of the way you encode mesh informations (connectivity table, block of elements, edges, vertices) in hdf5 files, and of the xdmf files that describe that layout? As I said in a previous message, I find the xdmf format documentation quite lacking. I also use Sieve to represent my mesh. Regards, Blaise -- Sent from a handheld device. On Jul 7, 2011, at 8:31 PM, Brad Aagaard wrote: > Ethan- > > I think writing an Xdmf file that points to the datasets in an HDF5 file is probably easier than creating a viewer. For PyLith we write the HDF5 file and a corresponding Xdmf file. This allows us to simply open the Xdmf file in ParaView; I think VisIt supports this as well. > > Brad > > > On 07/07/2011 10:12 AM, Ethan Coon wrote: >> The downside of this option is that, in order for hdf5 files to work in >> anything but python (which is smart enough to use introspection on the >> file itself) you need a special viewer (the VisIt people call this a >> "flavor" of HDF5), see >> >> http://visitusers.org/index.php?title=VisIt_and_HDF5 >> >> These work on both VisIt and Paraview, but I've never tried to write one >> myself, so I don't know how ugly they are. They basically specify the >> layout of the hdf5 file. >> >> Note that if you go with the HDF5 option, you'll likely want to move to >> petsc-dev, which includes more HDF5 functionality, such as groups. > From baagaard at usgs.gov Thu Jul 7 15:23:25 2011 From: baagaard at usgs.gov (Brad Aagaard) Date: Thu, 07 Jul 2011 13:23:25 -0700 Subject: [petsc-users] PETSc recommended visualization packages: Xdmf and HDF5 In-Reply-To: <15D137AD-7861-4C61-8876-AE816C45B9DF@lsu.edu> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <20110707082107.AJC19221@batman.int.colorado.edu> <1310058760.24206.54.camel@echo.lanl.gov> <4E15FB6F.4000704@usgs.gov> <15D137AD-7861-4C61-8876-AE816C45B9DF@lsu.edu> Message-ID: <4E1615BD.3090400@usgs.gov> Blaise- I constructed my Xdmf writer based on the documentation at http://www.xdmf.org/index.php/XDMF_Model_and_Format. The documentation is vague but I was able to figure most things out with a little trial and error. An example pair of HDF5 and Xdmf files is attached. The code is in the PyLith repository available at http://www.geodynamics.org/wsvn/cig/short/3D/PyLith/trunk/libsrc/pylith/meshio/?rev=18708&sc=0 The Xdmf object takes an HDF5 file (that we wrote so we know its layout) and writes the corresponding Xdmf file. The Xdmf.hh, Xdmf.cc, HDF5.hh, and HDF5.cc files are all in this directory. Regards, Brad On 07/07/2011 01:01 PM, Blaise A Bourdin wrote: > Brad, > > Do you have examples of the way you encode mesh informations > (connectivity table, block of elements, edges, vertices) in hdf5 > files, and of the xdmf files that describe that layout? As I said in > a previous message, I find the xdmf format documentation quite > lacking. > > I also use Sieve to represent my mesh. > > Regards, > > Blaise > -------------- next part -------------- A non-text attachment was scrubbed... Name: step06.h5.gz Type: application/gzip Size: 68990 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: step06.xmf Type: text/xml Size: 13970 bytes Desc: not available URL: From knepley at gmail.com Thu Jul 7 16:32:28 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 7 Jul 2011 21:32:28 +0000 Subject: [petsc-users] scattering and communications In-Reply-To: <4E15E515.6020806@polito.it> References: <4E15E515.6020806@polito.it> Message-ID: On Thu, Jul 7, 2011 at 4:55 PM, Marco Cisternino wrote: > Hi, > I would like to understand better how VecScatterBegin and VecScatterEnd > work. > I solve an interface elliptic problem on a Cartesian grid introducing extra > unknowns (intersections between the interface and the grid axes). > Therefore, my linear system is augmented with extra condition on these new > unknowns. > I use a DA to manage the grid, but I have to use MatCreateMPIAIJ to build > the matrix because of the nature of the problem. > MatCreateMPIAIJ(proc->cart_**comm,proc->intersections[proc-** > >rank],proc->intersections[**proc->rank],g_rows,g_cols,21,** > PETSC_NULL,21,PETSC_NULL,&**fsolv->AA); > VecCreateMPI(proc->cart_comm,**proc->intersections[proc->** > rank],PETSC_DECIDE,&fsolv->**extended_P); > VecCreateMPI(proc->cart_comm,**proc->intersections[proc->** > rank],PETSC_DECIDE,&fsolv->**extended_RHS); > > where > proc->intersections[proc->**rank] is the total number of unknowns for each > processor in its sub-domain (grid points + intersections). > g_rows=g_cols is the total number of unknowns in the entire computational > domain (grid points + intersections). > cart_comm is a Cartesian communicator. > > The arrangement of the unknowns is such that every processor has the rows > of the matrix and of extended_P(the solution) relative to the actual > unknowns in its sub-domain. > I solve the system and then I call VecScatterBegin and VecScatterEnd: > > ierr=VecScatterCreate(fsolv->**extended_P,scatter->from,vec->** > GP,scatter->to,&scatter->**scatt); > ierr=VecScatterBegin(scatter->**scatt,fsolv->extended_P,vec->** > GP,INSERT_VALUES,SCATTER_**FORWARD); > ierr=VecScatterEnd(scatter->**scatt,fsolv->extended_P,vec->** > GP,INSERT_VALUES,SCATTER_**FORWARD); > > in order to get in GP (made using DACreateGlobalVector(grid->da,**&vec->GP); > ) only the solution on the grid points. > It works, I mean I can get the right solution in GP, but the scattering > doesn't scale at all! > I would expect no communications during the scattering, doing what I do, > but -log_summary shows me a number of MPI message growing with the number of > processors. > Every portion of GP contains only the grid points of a processor, while > every portion of extended_P contains the same grid points plus the > intersections in the relative sub-domain. which is the need for the > communications doing such a scattering? > There is just a mismatch somewhere in your indices. It should be easy to check the locally owned indices in GP using http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Vec/VecGetOwnershipRange.html and compare to what you have. Matt > I don't know if I was clear enough. Please, ask me what you need to > understand my problem. > Thanks a lot. > > Best regards, > > Marco > > -- > Marco Cisternino > PhD Student > Politecnico di Torino > Mobile:+393281189696 > Email:marco.cisternino at polito.**it > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Fri Jul 8 07:27:16 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Fri, 08 Jul 2011 14:27:16 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! Message-ID: <4E16F7A4.3060200@gmail.com> Hi, I am trying to solve the momentum equation. The 1st time step was ok, but at the 2nd, the error message is: Petsc has generated inconsistent data - Divide by zero! What does it mean? Do you know what is the likely cause of the error? I did not find an unusually large values in matrix and RHS vector. Thank you very much. -- Yours sincerely, TAY wee-beng From jedbrown at mcs.anl.gov Fri Jul 8 07:37:56 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 8 Jul 2011 07:37:56 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E16F7A4.3060200@gmail.com> References: <4E16F7A4.3060200@gmail.com> Message-ID: On Fri, Jul 8, 2011 at 07:27, TAY wee-beng wrote: > I am trying to solve the momentum equation. > fluids, solids, particles? > The 1st time step was ok, but at the 2nd, the error message is: > > Petsc has generated inconsistent data - Divide by zero! > Always send the full error output. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Fri Jul 8 08:26:16 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Fri, 08 Jul 2011 15:26:16 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> Message-ID: <4E170578.2030209@gmail.com> Hi Jed, Sorry for the insufficient information. I am solving the NS eqns, fluids. The full error output is: [0]PETSC ERROR: --------------------- Error Message ---------------------------- -------- [0]PETSC ERROR: Petsc has generated inconsistent data! [0]PETSC ERROR: Divide by zero! [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 10 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: C:\Obj_tmp\ibm2d_high_Re_wo_hypre_RK3\Debug\ibm2d_high_Re_wo_hyp re.exe on a petsc-3.1 named EXPERIEN-33372E by Administrator Fri Jul 08 14:11:50 2011 [0]PETSC ERROR: Libraries linked from /cygdrive/c/Libs/petsc-3.1-p7_win32_cvf/li b [0]PETSC ERROR: Configure run at Thu Feb 10 02:42:16 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc="win32fe f90" --download-f-blas-lapack=1 --with-mpi-dir=/cygdrive/c/Libs/MPICH2/ --prefix=/cy gdrive/c/Libs/petsc-3.1-p7_win32_cvf --with-debugging=1 --useThreads=0 [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: KSPSolve_BCGS() line 75 in src/ksp/ksp/impls/bcgs/C:\Codes\PETSC -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c [0]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/C:\Codes\PETSC-~1.1 -P\src\ksp\ksp\INTERF~1\itfunc.c [0]PETSC ERROR: --------------------- Error Message ---------------------------- -------- [0]PETSC ERROR: Petsc has generated inconsistent data! [0]PETSC ERROR: Divide by zero! [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 10 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: C:\Obj_tmp\ibm2d_high_Re_wo_hypre_RK3\Debug\ibm2d_high_Re_wo_hyp re.exe on a petsc-3.1 named EXPERIEN-33372E by Administrator Fri Jul 08 14:11:50 2011 [0]PETSC ERROR: Libraries linked from /cygdrive/c/Libs/petsc-3.1-p7_win32_cvf/li b [0]PETSC ERROR: Configure run at Thu Feb 10 02:42:16 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc="win32fe f90" --download-f-blas-lapack=1 --with-mpi-dir=/cygdrive/c/Libs/MPICH2/ --prefix=/cy gdrive/c/Libs/petsc-3.1-p7_win32_cvf --with-debugging=1 --useThreads=0 [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: KSPSolve_BCGS() line 75 in src/ksp/ksp/impls/bcgs/C:\Codes\PETSC -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c [0]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/C:\Codes\PETSC-~1.1 -P\src\ksp\ksp\INTERF~1\itfunc.c [0]PETSC ERROR: --------------------- Error Message ---------------------------- -------- [0]PETSC ERROR: Petsc has generated inconsistent data! [0]PETSC ERROR: Divide by zero! [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 10 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: C:\Obj_tmp\ibm2d_high_Re_wo_hypre_RK3\Debug\ibm2d_high_Re_wo_hyp re.exe on a petsc-3.1 named EXPERIEN-33372E by Administrator Fri Jul 08 14:11:50 2011 [0]PETSC ERROR: Libraries linked from /cygdrive/c/Libs/petsc-3.1-p7_win32_cvf/li b [0]PETSC ERROR: Configure run at Thu Feb 10 02:42:16 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc="win32fe f90" --download-f-blas-lapack=1 --with-mpi-dir=/cygdrive/c/Libs/MPICH2/ --prefix=/cy gdrive/c/Libs/petsc-3.1-p7_win32_cvf --with-debugging=1 --useThreads=0 [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: KSPSolve_BCGS() line 75 in src/ksp/ksp/impls/bcgs/C:\Codes\PETSC -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c [0]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/C:\Codes\PETSC-~1.1 -P\src\ksp\ksp\INTERF~1\itfunc.c [0]PETSC ERROR: --------------------- Error Message ---------------------------- -------- [0]PETSC ERROR: Petsc has generated inconsistent data! [0]PETSC ERROR: Divide by zero! [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 10 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: C:\Obj_tmp\ibm2d_high_Re_wo_hypre_RK3\Debug\ibm2d_high_Re_wo_hyp re.exe on a petsc-3.1 named EXPERIEN-33372E by Administrator Fri Jul 08 14:11:50 2011 [0]PETSC ERROR: Libraries linked from /cygdrive/c/Libs/petsc-3.1-p7_win32_cvf/li b [0]PETSC ERROR: Configure run at Thu Feb 10 02:42:16 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc="win32fe f90" --download-f-blas-lapack=1 --with-mpi-dir=/cygdrive/c/Libs/MPICH2/ --prefix=/cy gdrive/c/Libs/petsc-3.1-p7_win32_cvf --with-debugging=1 --useThreads=0 [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: KSPSolve_BCGS() line 75 in src/ksp/ksp/impls/bcgs/C:\Codes\PETSC -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c [0]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/C:\Codes\PETSC-~1.1 -P\src\ksp\ksp\INTERF~1\itfunc.c u,v exploded! Hope this helps. Yours sincerely, TAY wee-beng On 8/7/2011 2:37 PM, Jed Brown wrote: > On Fri, Jul 8, 2011 at 07:27, TAY wee-beng > wrote: > > I am trying to solve the momentum equation. > > > fluids, solids, particles? > > The 1st time step was ok, but at the 2nd, the error message is: > > Petsc has generated inconsistent data - Divide by zero! > > > Always send the full error output. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Jul 8 08:34:23 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 8 Jul 2011 08:34:23 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E170578.2030209@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> Message-ID: On Fri, Jul 8, 2011 at 08:26, TAY wee-beng wrote: > [0]PETSC ERROR: KSPSolve_BCGS() line 75 in > src/ksp/ksp/impls/bcgs/C:\Codes\PETSC > -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c > ierr = KSP_PCApplyBAorAB(ksp,P,V,T);CHKERRQ(ierr); /* v <- K p */ ierr = VecDot(V,RP,&d1);CHKERRQ(ierr); if (d1 == 0.0) SETERRQ(PETSC_ERR_PLIB,"Divide by zero"); I suspect your preconditioner is singular. What options are you running with? What happens if you use -ksp_type gmres -ksp_gmres_restart 1000 -ksp_gmres_modifiedgramschmidt -ksp_monitor_true_residual? Also do the same run with -ksp_type fgmres. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.cisternino at polito.it Fri Jul 8 10:48:15 2011 From: marco.cisternino at polito.it (Marco Cisternino) Date: Fri, 08 Jul 2011 17:48:15 +0200 Subject: [petsc-users] scattering and communications In-Reply-To: References: Message-ID: <4E1726BF.5040706@polito.it> Thanks for the reply, Matt. I tried what you suggested, but I saw no inconsistency in the portion given to GP or extended_P. Moreover, I tried to see what happens giving no interface, i.e. a simple Laplace equation. In this case GP and extended_P have exactly the same structure and partitioning. And nothing changes. All the communications happens in VecScatterCreate, the actual scatter (begin and end) uses almost no time and no communications, but VecScatterCreate does. Could you explain me why? The two IS I use to create the scatter context are from sequential c vectors as large as the entire computational domain. Could be this the problem? If I used smaller and local vectors to create the ISs and then the scatter context, would VecScatterCreate be more performing?? Thanks a lot. Marco > On Thu, Jul 7, 2011 at 4:55 PM, Marco Cisternino> wrote: >> Hi, >> I would like to understand better how VecScatterBegin and VecScatterEnd >> work. >> I solve an interface elliptic problem on a Cartesian grid introducing extra >> unknowns (intersections between the interface and the grid axes). >> Therefore, my linear system is augmented with extra condition on these new >> unknowns. >> I use a DA to manage the grid, but I have to use MatCreateMPIAIJ to build >> the matrix because of the nature of the problem. >> MatCreateMPIAIJ(proc->cart_**comm,proc->intersections[proc-** >>> rank],proc->intersections[**proc->rank],g_rows,g_cols,21,** >> PETSC_NULL,21,PETSC_NULL,&**fsolv->AA); >> VecCreateMPI(proc->cart_comm,**proc->intersections[proc->** >> rank],PETSC_DECIDE,&fsolv->**extended_P); >> VecCreateMPI(proc->cart_comm,**proc->intersections[proc->** >> rank],PETSC_DECIDE,&fsolv->**extended_RHS); >> >> where >> proc->intersections[proc->**rank] is the total number of unknowns for each >> processor in its sub-domain (grid points + intersections). >> g_rows=g_cols is the total number of unknowns in the entire computational >> domain (grid points + intersections). >> cart_comm is a Cartesian communicator. >> >> The arrangement of the unknowns is such that every processor has the rows >> of the matrix and of extended_P(the solution) relative to the actual >> unknowns in its sub-domain. >> I solve the system and then I call VecScatterBegin and VecScatterEnd: >> >> ierr=VecScatterCreate(fsolv->**extended_P,scatter->from,vec->** >> GP,scatter->to,&scatter->**scatt); >> ierr=VecScatterBegin(scatter->**scatt,fsolv->extended_P,vec->** >> GP,INSERT_VALUES,SCATTER_**FORWARD); >> ierr=VecScatterEnd(scatter->**scatt,fsolv->extended_P,vec->** >> GP,INSERT_VALUES,SCATTER_**FORWARD); >> >> in order to get in GP (made using DACreateGlobalVector(grid->da,**&vec->GP); >> ) only the solution on the grid points. >> It works, I mean I can get the right solution in GP, but the scattering >> doesn't scale at all! >> I would expect no communications during the scattering, doing what I do, >> but -log_summary shows me a number of MPI message growing with the number of >> processors. >> Every portion of GP contains only the grid points of a processor, while >> every portion of extended_P contains the same grid points plus the >> intersections in the relative sub-domain. which is the need for the >> communications doing such a scattering? >> > There is just a mismatch somewhere in your indices. It should be easy to > check the locally owned indices in > GP using > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Vec/VecGetOwnershipRange.html > > and compare to what you have. > > Matt > > >> I don't know if I was clear enough. Please, ask me what you need to >> understand my problem. >> Thanks a lot. >> >> Best regards, >> >> Marco >> >> -- >> Marco Cisternino >> PhD Student >> Politecnico di Torino >> Mobile:+393281189696 >> Email:marco.cisternino at polito.**it >> >> > -- Marco Cisternino PhD Student Politecnico di Torino Mobile:+393281189696 Email:marco.cisternino at polito.it From john.fettig at gmail.com Fri Jul 8 13:31:17 2011 From: john.fettig at gmail.com (John Fettig) Date: Fri, 8 Jul 2011 14:31:17 -0400 Subject: [petsc-users] Controlling ML Message-ID: Is there any way to control ML other than the options database? For example, I want to set -pc_ml_maxNlevels 3, but there seems to be no way to do it other than a) the command line or b) PetscOptionsSetValue("-pc_ml_maxNlevels","3"); This doesn't make it easy if you want to have more than one PCML. John From balay at mcs.anl.gov Fri Jul 8 13:39:25 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 8 Jul 2011 13:39:25 -0500 (CDT) Subject: [petsc-users] Controlling ML In-Reply-To: References: Message-ID: looks like there is no interface function to set this option. Alternative: use optionsprefix to distuingish diffrent solves KSPSetOptionsPrefix(ksp1,"solver1_") KSPSetOptionsPrefix(ksp2,"solver2_") PetscOptionsSetValue("-solver1_pc_ml_maxNlevels","3"); PetscOptionsSetValue("-solver2_pc_ml_maxNlevels","4"); etc.. Satish On Fri, 8 Jul 2011, John Fettig wrote: > Is there any way to control ML other than the options database? For > example, I want to set -pc_ml_maxNlevels 3, but there seems to be no > way to do it other than > > a) the command line or > b) PetscOptionsSetValue("-pc_ml_maxNlevels","3"); > > This doesn't make it easy if you want to have more than one PCML. > > John > From jedbrown at mcs.anl.gov Fri Jul 8 13:39:58 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 8 Jul 2011 13:39:58 -0500 Subject: [petsc-users] Controlling ML In-Reply-To: References: Message-ID: There are not setters fo every PCML option. Regardless, you should set a different prefix for each solver, then you can control them independently. On Jul 8, 2011 1:31 PM, "John Fettig" wrote: > Is there any way to control ML other than the options database? For > example, I want to set -pc_ml_maxNlevels 3, but there seems to be no > way to do it other than > > a) the command line or > b) PetscOptionsSetValue("-pc_ml_maxNlevels","3"); > > This doesn't make it easy if you want to have more than one PCML. > > John -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.fettig at gmail.com Fri Jul 8 13:41:35 2011 From: john.fettig at gmail.com (John Fettig) Date: Fri, 8 Jul 2011 14:41:35 -0400 Subject: [petsc-users] Controlling ML In-Reply-To: References: Message-ID: Ok, I will do that. Thanks for the quick response! John On Fri, Jul 8, 2011 at 2:39 PM, Jed Brown wrote: > There are not setters fo every PCML option. Regardless, you should set a > different prefix for each solver, then you can control them independently. > > On Jul 8, 2011 1:31 PM, "John Fettig" wrote: >> Is there any way to control ML other than the options database? For >> example, I want to set -pc_ml_maxNlevels 3, but there seems to be no >> way to do it other than >> >> a) the command line or >> b) PetscOptionsSetValue("-pc_ml_maxNlevels","3"); >> >> This doesn't make it easy if you want to have more than one PCML. >> >> John > From knepley at gmail.com Fri Jul 8 16:45:22 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 8 Jul 2011 21:45:22 +0000 Subject: [petsc-users] scattering and communications In-Reply-To: <4E1726BF.5040706@polito.it> References: <4E1726BF.5040706@polito.it> Message-ID: On Fri, Jul 8, 2011 at 3:48 PM, Marco Cisternino wrote: > Thanks for the reply, Matt. > I tried what you suggested, but I saw no inconsistency in the portion given > to GP or extended_P. > Moreover, I tried to see what happens giving no interface, i.e. a simple > Laplace equation. > In this case GP and extended_P have exactly the same structure and > partitioning. And nothing changes. > All the communications happens in VecScatterCreate, the actual scatter > (begin and end) uses almost no time and no communications, but > VecScatterCreate does. > Could you explain me why? > VecScatterCreate() must find out what needs to be communicated, even if that answer is nothing needs to be communicated. Matt > The two IS I use to create the scatter context are from sequential c > vectors as large as the entire computational domain. Could be this the > problem? > If I used smaller and local vectors to create the ISs and then the scatter > context, would VecScatterCreate be more performing?? > Thanks a lot. > > Marco > > > On Thu, Jul 7, 2011 at 4:55 PM, Marco Cisternino> polito.it >> >>> wrote: >>> Hi, >>> I would like to understand better how VecScatterBegin and VecScatterEnd >>> work. >>> I solve an interface elliptic problem on a Cartesian grid introducing >>> extra >>> unknowns (intersections between the interface and the grid axes). >>> Therefore, my linear system is augmented with extra condition on these >>> new >>> unknowns. >>> I use a DA to manage the grid, but I have to use MatCreateMPIAIJ to build >>> the matrix because of the nature of the problem. >>> MatCreateMPIAIJ(proc->cart_****comm,proc->intersections[proc-**** >>> >>>> rank],proc->intersections[****proc->rank],g_rows,g_cols,21,**** >>>> >>> PETSC_NULL,21,PETSC_NULL,&****fsolv->AA); >>> VecCreateMPI(proc->cart_comm,****proc->intersections[proc->** >>> rank],PETSC_DECIDE,&fsolv->****extended_P); >>> VecCreateMPI(proc->cart_comm,****proc->intersections[proc->** >>> rank],PETSC_DECIDE,&fsolv->****extended_RHS); >>> >>> where >>> proc->intersections[proc->****rank] is the total number of unknowns for >>> each >>> processor in its sub-domain (grid points + intersections). >>> g_rows=g_cols is the total number of unknowns in the entire computational >>> domain (grid points + intersections). >>> cart_comm is a Cartesian communicator. >>> >>> The arrangement of the unknowns is such that every processor has the rows >>> of the matrix and of extended_P(the solution) relative to the actual >>> unknowns in its sub-domain. >>> I solve the system and then I call VecScatterBegin and VecScatterEnd: >>> >>> ierr=VecScatterCreate(fsolv->****extended_P,scatter->from,vec-**>** >>> GP,scatter->to,&scatter->****scatt); >>> ierr=VecScatterBegin(scatter->****scatt,fsolv->extended_P,vec-**>** >>> GP,INSERT_VALUES,SCATTER_****FORWARD); >>> ierr=VecScatterEnd(scatter->****scatt,fsolv->extended_P,vec->**** >>> GP,INSERT_VALUES,SCATTER_****FORWARD); >>> >>> in order to get in GP (made using DACreateGlobalVector(grid->da,** >>> **&vec->GP); >>> ) only the solution on the grid points. >>> It works, I mean I can get the right solution in GP, but the scattering >>> doesn't scale at all! >>> I would expect no communications during the scattering, doing what I do, >>> but -log_summary shows me a number of MPI message growing with the number >>> of >>> processors. >>> Every portion of GP contains only the grid points of a processor, while >>> every portion of extended_P contains the same grid points plus the >>> intersections in the relative sub-domain. which is the need for the >>> communications doing such a scattering? >>> >>> There is just a mismatch somewhere in your indices. It should be easy to >> check the locally owned indices in >> GP using >> >> >> http://www.mcs.anl.gov/petsc/**petsc-as/snapshots/petsc-dev/** >> docs/manualpages/Vec/**VecGetOwnershipRange.html >> >> and compare to what you have. >> >> Matt >> >> >> I don't know if I was clear enough. Please, ask me what you need to >>> understand my problem. >>> Thanks a lot. >>> >>> Best regards, >>> >>> Marco >>> >>> -- >>> Marco Cisternino >>> PhD Student >>> Politecnico di Torino >>> Mobile:+393281189696 >>> Email:marco.cisternino at polito.****it>> polito.it > >>> >>> >>> >> > -- > Marco Cisternino > PhD Student > Politecnico di Torino > Mobile:+393281189696 > Email:marco.cisternino at polito.**it > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Fri Jul 8 17:17:02 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sat, 09 Jul 2011 00:17:02 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> Message-ID: <4E1781DE.7070207@gmail.com> Hi Jed, This is my output when the error happens for gmres: 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm 2.58995443 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm 2.58995443 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm 9.05939430 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm 9.05939430 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm 9.05939430 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm 9.05939430 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm 9.05939430 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm 1.96385966 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm 1.96385966 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm 1.96385966 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm 1.96385966 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm 1.96385966 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 for fgmres: 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm 3.35941610 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm 3.35941610 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm 3.88294769 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm 3.88294769 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm 3.88294769 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm 3.88294769 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm 3.88294769 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm 3.89282805 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm 3.89282805 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm 3.89282805 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm 3.89282805 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm 3.89282805 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 Btw, I am solving the x and y momentum eqns so there 's 2 eqns to solve. I am using windows so I can show only part of the output. If req, I can transfer my code to linux and capture the output. Hope this explains something. Thanks! Yours sincerely, TAY wee-beng On 8/7/2011 3:34 PM, Jed Brown wrote: > On Fri, Jul 8, 2011 at 08:26, TAY wee-beng > wrote: > > [0]PETSC ERROR: KSPSolve_BCGS() line 75 in > src/ksp/ksp/impls/bcgs/C:\Codes\PETSC > -~1.1-P\src\ksp\ksp\impls\bcgs\bcgs.c > > > ierr = KSP_PCApplyBAorAB(ksp,P,V,T);CHKERRQ(ierr); /* v <- K p > */ > ierr = VecDot(V,RP,&d1);CHKERRQ(ierr); > if (d1 == 0.0) SETERRQ(PETSC_ERR_PLIB,"Divide by zero"); > > I suspect your preconditioner is singular. What options are you > running with? What happens if you use -ksp_type gmres > -ksp_gmres_restart 1000 -ksp_gmres_modifiedgramschmidt > -ksp_monitor_true_residual? Also do the same run with -ksp_type fgmres. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Jul 8 17:26:17 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 8 Jul 2011 17:26:17 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E1781DE.7070207@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> Message-ID: On Fri, Jul 8, 2011 at 17:17, TAY wee-beng wrote: > ** > Hi Jed, > > This is my output when the error happens for gmres: > > 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm > 2.58995443 > 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 > 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm > 2.58995443 > 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm > 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm > 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid norm > 1.207952812560e+003 true resid norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm > 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm > 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm > 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm > 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm > 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm > 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm > 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > Why is the same line output multiple times? Do you have an MPI problem? This convergence looks funky; I suspect your preconditioner is singular and/or nonlinear. Please send -ksp_view output as you are running here and with -pc_type asm -sub_pc_type lu. > > for fgmres: > > 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm > 3.35941610 > 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 > 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm > 3.35941610 > 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm > 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm > 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm > 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm > 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm > 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm > 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm > 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm > 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm > 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm > 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > > Btw, I am solving the x and y momentum eqns so there 's 2 eqns to solve. > Do you solve them separately or both at once? (We usually recommend the latter.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Sat Jul 9 17:21:49 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sun, 10 Jul 2011 00:21:49 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> Message-ID: <4E18D47D.8060807@gmail.com> Hi Jed, I have attached the output runing under linux. It's only running with 1 processor. I am using runge kutta 3rd order to solve my problem, so there's 3 substeps per time step ie I need to solve the x/y momentum eqns 3 times per time step. The error occurs at t=2, substep=2. It stated in the output. Btw, I am solving the x and y momentum eqns separately, since they are independent of one another. How can I solve it together? My code is: call MatSetValues(A_semi_x,1,II,1,JJ,semi_mat_x(k,kk),INSERT_VALUES,ierr) ... call MatAssemblyBegin(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) call MatAssemblyEnd(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) call KSPGetPC(ksp_semi_x,pc_semi_x,ierr) ksptype=KSPBCGS call KSPSetType(ksp_semi_x,ksptype,ierr) call KSPSetFromOptions(ksp_semi_x,ierr) tol=1.e-5 call KSPSetTolerances(ksp_semi_x,tol,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,ierr) exactly the same for y... call VecAssemblyBegin(b_rhs_semi_x,ierr) call VecAssemblyEnd(b_rhs_semi_x,ierr) call VecAssemblyBegin(xx_semi_x,ierr) call VecAssemblyEnd(xx_semi_x,ierr) call KSPSetOperators(ksp_semi_x,A_semi_x,A_semi_x,SAME_NONZERO_PATTERN,ierr) call KSPSolve(ksp_semi_x,b_rhs_semi_x,xx_semi_x,ierr) call KSPGetConvergedReason(ksp_semi_x,reason,ierr) exactly the same for y... Thanks alot! Yours sincerely, TAY wee-beng On 9/7/2011 12:26 AM, Jed Brown wrote: > On Fri, Jul 8, 2011 at 17:17, TAY wee-beng > wrote: > > Hi Jed, > > This is my output when the error happens for gmres: > > 2 KSP preconditioned resid norm 1.556688898463e+007 true resid > norm 2.58995443 > 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 > 2 KSP preconditioned resid norm 1.556688898463e+007 true resid > norm 2.58995443 > 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid > norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid > norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned > resid norm 1.207952812560e+003 true resid norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid > norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 3 KSP preconditioned resid norm 1.207952812560e+003 true resid > norm 9.05939430 > 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid > norm 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid > norm 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid > norm 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid > norm 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > 4 KSP preconditioned resid norm 3.815055004710e+001 true resid > norm 1.96385966 > 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 > > > Why is the same line output multiple times? Do you have an MPI problem? > > This convergence looks funky; I suspect your preconditioner is > singular and/or nonlinear. Please send -ksp_view output as you are > running here and with -pc_type asm -sub_pc_type lu. > > > for fgmres: > > 34 KSP preconditioned resid norm 2.428114102327e-005 true resid > norm 3.35941610 > 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 > 34 KSP preconditioned resid norm 2.428114102327e-005 true resid > norm 3.35941610 > 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid > norm 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid > norm 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid > norm 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid > norm 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 35 KSP preconditioned resid norm 2.200715659545e-005 true resid > norm 3.88294769 > 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid > norm 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid > norm 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid > norm 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid > norm 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > 36 KSP preconditioned resid norm 2.011575409369e-005 true resid > norm 3.89282805 > 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 > > Btw, I am solving the x and y momentum eqns so there 's 2 eqns to > solve. > > > Do you solve them separately or both at once? (We usually recommend > the latter.) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: log1 URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: log2 URL: From knepley at gmail.com Sat Jul 9 17:26:29 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 9 Jul 2011 22:26:29 +0000 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E18D47D.8060807@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> Message-ID: 2011/7/9 TAY wee-beng > ** > Hi Jed, > > I have attached the output runing under linux. It's only running with 1 > processor. I am using runge kutta 3rd order to solve my problem, so there's > 3 substeps per time step ie I need to solve the x/y momentum eqns 3 times > per time step. > 1) As Jed said, you have repeated output. Something is wrong with your MPI setup. I suspect you are using the wrong mpiexec. Please do not send parallel results until your serial results work. 2) Always run with -ksp_view when you send results 3) You did not try using LU as Jed suggested. Matt > The error occurs at t=2, substep=2. It stated in the output. > > Btw, I am solving the x and y momentum eqns separately, since they are > independent of one another. How can I solve it together? > > My code is: > > call MatSetValues(A_semi_x,1,II,1,JJ,semi_mat_x(k,kk),INSERT_VALUES,ierr) > ... > > call MatAssemblyBegin(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) > call MatAssemblyEnd(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) > > call KSPGetPC(ksp_semi_x,pc_semi_x,ierr) > ksptype=KSPBCGS > call KSPSetType(ksp_semi_x,ksptype,ierr) > call KSPSetFromOptions(ksp_semi_x,ierr) > > tol=1.e-5 > > call > KSPSetTolerances(ksp_semi_x,tol,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,ierr) > > exactly the same for y... > > call VecAssemblyBegin(b_rhs_semi_x,ierr) > call VecAssemblyEnd(b_rhs_semi_x,ierr) > > call VecAssemblyBegin(xx_semi_x,ierr) > call VecAssemblyEnd(xx_semi_x,ierr) > > call > KSPSetOperators(ksp_semi_x,A_semi_x,A_semi_x,SAME_NONZERO_PATTERN,ierr) > call KSPSolve(ksp_semi_x,b_rhs_semi_x,xx_semi_x,ierr) > call KSPGetConvergedReason(ksp_semi_x,reason,ierr) > > exactly the same for y... > > Thanks alot! > > Yours sincerely, > > TAY wee-beng > > > On 9/7/2011 12:26 AM, Jed Brown wrote: > > On Fri, Jul 8, 2011 at 17:17, TAY wee-beng wrote: > >> Hi Jed, >> >> This is my output when the error happens for gmres: >> >> 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm >> 2.58995443 >> 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 >> 2 KSP preconditioned resid norm 1.556688898463e+007 true resid norm >> 2.58995443 >> 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm >> 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm >> 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP preconditioned resid >> norm 1.207952812560e+003 true resid norm 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm >> 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true resid norm >> 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm >> 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm >> 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm >> 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm >> 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true resid norm >> 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> > > Why is the same line output multiple times? Do you have an MPI problem? > > This convergence looks funky; I suspect your preconditioner is singular > and/or nonlinear. Please send -ksp_view output as you are running here and > with -pc_type asm -sub_pc_type lu. > > >> >> for fgmres: >> >> 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm >> 3.35941610 >> 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 >> 34 KSP preconditioned resid norm 2.428114102327e-005 true resid norm >> 3.35941610 >> 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm >> 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm >> 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm >> 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm >> 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true resid norm >> 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm >> 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm >> 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm >> 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm >> 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true resid norm >> 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> >> Btw, I am solving the x and y momentum eqns so there 's 2 eqns to solve. >> > > Do you solve them separately or both at once? (We usually recommend the > latter.) > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Sat Jul 9 17:43:58 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sun, 10 Jul 2011 00:43:58 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> Message-ID: <4E18D9AE.3040106@gmail.com> Hi, Sorry about that. I just run interactively to get the output. Now I issued : mpiexec -np 1 ./a.out -ksp_type gmres -ksp_gmres_restart 1000 -ksp_gmres_modifiedgramschmidt -ksp_monitor_true_residual -ksp_view -pc_type asm -sub_pc_type lu > log1 Same for the fgmres. Hope it's ok now. Thanks! Yours sincerely, TAY wee-beng On 10/7/2011 12:26 AM, Matthew Knepley wrote: > 2011/7/9 TAY wee-beng > > > Hi Jed, > > I have attached the output runing under linux. It's only running > with 1 processor. I am using runge kutta 3rd order to solve my > problem, so there's 3 substeps per time step ie I need to solve > the x/y momentum eqns 3 times per time step. > > > 1) As Jed said, you have repeated output. Something is wrong with your > MPI setup. I suspect you are using > the wrong mpiexec. Please do not send parallel results until your > serial results work. > > 2) Always run with -ksp_view when you send results > > 3) You did not try using LU as Jed suggested. > > Matt > > The error occurs at t=2, substep=2. It stated in the output. > > Btw, I am solving the x and y momentum eqns separately, since they > are independent of one another. How can I solve it together? > > My code is: > > call > MatSetValues(A_semi_x,1,II,1,JJ,semi_mat_x(k,kk),INSERT_VALUES,ierr) > ... > > call MatAssemblyBegin(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) > call MatAssemblyEnd(A_semi_x,MAT_FINAL_ASSEMBLY,ierr) > > call KSPGetPC(ksp_semi_x,pc_semi_x,ierr) > ksptype=KSPBCGS > call KSPSetType(ksp_semi_x,ksptype,ierr) > call KSPSetFromOptions(ksp_semi_x,ierr) > > tol=1.e-5 > > call > KSPSetTolerances(ksp_semi_x,tol,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,ierr) > > exactly the same for y... > > call VecAssemblyBegin(b_rhs_semi_x,ierr) > call VecAssemblyEnd(b_rhs_semi_x,ierr) > > call VecAssemblyBegin(xx_semi_x,ierr) > call VecAssemblyEnd(xx_semi_x,ierr) > > call > KSPSetOperators(ksp_semi_x,A_semi_x,A_semi_x,SAME_NONZERO_PATTERN,ierr) > call KSPSolve(ksp_semi_x,b_rhs_semi_x,xx_semi_x,ierr) > call KSPGetConvergedReason(ksp_semi_x,reason,ierr) > > exactly the same for y... > > Thanks alot! > > Yours sincerely, > > TAY wee-beng > > > On 9/7/2011 12:26 AM, Jed Brown wrote: >> On Fri, Jul 8, 2011 at 17:17, TAY wee-beng > > wrote: >> >> Hi Jed, >> >> This is my output when the error happens for gmres: >> >> 2 KSP preconditioned resid norm 1.556688898463e+007 true >> resid norm 2.58995443 >> 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 >> 2 KSP preconditioned resid norm 1.556688898463e+007 true >> resid norm 2.58995443 >> 8801e+003 ||Ae||/||Ax|| 1.232088736212e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true >> resid norm 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true >> resid norm 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 3 KSP >> preconditioned resid norm 1.207952812560e+003 true resid norm >> 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true >> resid norm 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 3 KSP preconditioned resid norm 1.207952812560e+003 true >> resid norm 9.05939430 >> 1179e+003 ||Ae||/||Ax|| 4.309719703237e+003 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true >> resid norm 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true >> resid norm 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true >> resid norm 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true >> resid norm 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> 4 KSP preconditioned resid norm 3.815055004710e+001 true >> resid norm 1.96385966 >> 1729e+002 ||Ae||/||Ax|| 9.342439899591e+001 >> >> >> Why is the same line output multiple times? Do you have an MPI >> problem? >> >> This convergence looks funky; I suspect your preconditioner is >> singular and/or nonlinear. Please send -ksp_view output as you >> are running here and with -pc_type asm -sub_pc_type lu. >> >> >> for fgmres: >> >> 34 KSP preconditioned resid norm 2.428114102327e-005 true >> resid norm 3.35941610 >> 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 >> 34 KSP preconditioned resid norm 2.428114102327e-005 true >> resid norm 3.35941610 >> 6265e-005 ||Ae||/||Ax|| 1.598129588716e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true >> resid norm 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true >> resid norm 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true >> resid norm 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true >> resid norm 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 35 KSP preconditioned resid norm 2.200715659545e-005 true >> resid norm 3.88294769 >> 5703e-005 ||Ae||/||Ax|| 1.847182191086e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true >> resid norm 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true >> resid norm 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true >> resid norm 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true >> resid norm 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> 36 KSP preconditioned resid norm 2.011575409369e-005 true >> resid norm 3.89282805 >> 6204e-005 ||Ae||/||Ax|| 1.851882441357e-005 >> >> Btw, I am solving the x and y momentum eqns so there 's 2 >> eqns to solve. >> >> >> Do you solve them separately or both at once? (We usually >> recommend the latter.) > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: log_fgmres URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: log_gmres URL: From jedbrown at mcs.anl.gov Sat Jul 9 17:51:16 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 9 Jul 2011 17:51:16 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E18D9AE.3040106@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> <4E18D9AE.3040106@gmail.com> Message-ID: On Sat, Jul 9, 2011 at 17:43, TAY wee-beng wrote: > Sorry about that. I just run interactively to get the output. > > Now I issued : > > mpiexec -np 1 ./a.out -ksp_type gmres -ksp_gmres_restart 1000 > -ksp_gmres_modifiedgramschmidt -ksp_monitor_true_residual -ksp_view -pc_type > asm -sub_pc_type lu > log1 > Your matrix is singular which implies a bug in your assembly code, probably in the way you impose boundary conditions. Switch to a very small problem size (e.g. 4 cells) and debug it until the matrix is correct and non-singular. You should also check the error codes so the code doesn't try to keep going after a fatal error. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Sat Jul 9 17:57:51 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sun, 10 Jul 2011 00:57:51 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> <4E18D9AE.3040106@gmail.com> Message-ID: <4E18DCEF.6020107@gmail.com> Hi Jed, Ok thanks. Btw, is it better to solve my x/y eqns together? How can I do that? Is there any examples which I can follow? Yours sincerely, TAY wee-beng On 10/7/2011 12:51 AM, Jed Brown wrote: > On Sat, Jul 9, 2011 at 17:43, TAY wee-beng > wrote: > > Sorry about that. I just run interactively to get the output. > > Now I issued : > > mpiexec -np 1 ./a.out -ksp_type gmres -ksp_gmres_restart 1000 > -ksp_gmres_modifiedgramschmidt -ksp_monitor_true_residual > -ksp_view -pc_type asm -sub_pc_type lu > log1 > > > Your matrix is singular which implies a bug in your assembly code, > probably in the way you impose boundary conditions. Switch to a very > small problem size (e.g. 4 cells) and debug it until the matrix is > correct and non-singular. > > You should also check the error codes so the code doesn't try to keep > going after a fatal error. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Jul 9 18:02:30 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 9 Jul 2011 18:02:30 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E18DCEF.6020107@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> <4E18D9AE.3040106@gmail.com> <4E18DCEF.6020107@gmail.com> Message-ID: On Sat, Jul 9, 2011 at 17:57, TAY wee-beng wrote: > Ok thanks. Btw, is it better to solve my x/y eqns together? How can I do > that? > Sure, interlace the x and y degrees of freedom and assemble the matrix with x equations in the even row/column numbers and y equations in the odd row/column numbers. Note that the equations are not decoupled for some boundary conditions (e.g. slip on a curved surface or free surface). -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Sun Jul 10 05:44:10 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Sun, 10 Jul 2011 06:44:10 -0400 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: Message-ID: 1) Since Linux now (as of the 2.6 kernel) supports a 1-to-1 association/mapping between pthreads (user threads) and kthreads (kernel threads), plus the new process scheduler, this makes pthreads much more efficient/useful for MIMD parallelism. I assume this makes a thread-based solution (PETSc) more attractive? When PETSc was first conceived, this was not the case. 2) Since threads can share data (variables), wouldn't there be less of a need to send/receive data than using MPI, which simply spawns separate disjoint processes? 3) " Vec and Mat class"? I hope you're sticking with C (v. C++)? I've used G++ version 4.3.4 and version 4.5.2 and got different answers with the exact same source. I have used Visual C++ (2008) and got different answers than G++ 4.6.1 - again the same source. C99 seems to give consistent results across compilers/platforms and is simpler to boot. ---John On Sat, Jul 9, 2011 at 11:41 PM, Barry Smith wrote: > > We have started a pthread based Vec and Mat class that lives inside the > PETSc MPI world. We will be pushing it into the petsc-dev repository in the > next could of weeks. > > Notes: > > 1) Since all the numerical computations take place in the Vec and Mat class > those are the ones that need to use pthreads underneath, the other classes > like KSP do not. > 2) A group in England has developed a OpenMP based Vec and Mat class in the > same vain as our pthread version; I've been trying without success to get it > also into petsc-dev, hopefully I'll eventually charm them into it. > 3) Since sparse iterative solvers are very much memory bandwidth limited > (the time to solve is determined by the speed of the memory, not the CPUs) > there is only some much one can do to improve performance by throwing more > cores (threads) at the problem. For example most cheap desktop systems do > not have enough memory bandwidth to support even two cores doing the > iterative solver so adding pthreads will only matter on certain (more > expensive) systems and is not a "cure all". See > http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers > > > Barry > > > On Jul 9, 2011, at 10:31 PM, John Chludzinski wrote: > > > Have you guys considered a pthread based implementation of PETSc/SLEPc. > > Have you investigated the use of pthreads? ---John > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 10 08:13:49 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 08:13:49 -0500 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: Message-ID: On Sun, Jul 10, 2011 at 05:44, John Chludzinski wrote: > 2) Since threads can share data (variables), wouldn't there be less of a > need to send/receive data than using MPI, which simply spawns separate > disjoint processes? > Yes, threads allow more things to be shared, but there is still a problem of data locality and it is often faster to copy than to share. > > 3) " Vec and Mat class"? I hope you're sticking with C (v. C++)? I've > used G++ version 4.3.4 and version 4.5.2 and got different answers with the > exact same source. I have used Visual C++ (2008) and got different answers > than G++ 4.6.1 - again the same source. C99 seems to give consistent > results across compilers/platforms and is simpler to boot. > We generally prefer C to C++, but the symptoms you describe are likely to be caused by writing code with undefined behavior. Unfortunately, there is no checker for undefined behavior, but compiling with -std=c++98 -Wall -Wextra -pedantic does show many things. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Yudong.Sun at nag.co.uk Fri Jul 8 09:15:56 2011 From: Yudong.Sun at nag.co.uk (Yudong Sun) Date: Fri, 08 Jul 2011 15:15:56 +0100 Subject: [petsc-users] petsc-3.1-p8 configure error Message-ID: <4E17111C.6000502@nag.co.uk> I am configuring petsc-3.1-p8 for GNU compilers with the options as follows. ./configure --known-mpi-c-double-complex=0 --known-level1-dcache-size=65536 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=2 --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=8 --known-mpi-long-double=0 --with-x=0 --with-batch --with-blas-lib=-lsci --with-lapack-lib=-lsci --with-dynamic=false --with-shared=false --CXX=CC --CC=cc --CPP=cpp --with-cpp=cpp --FC=ftn --with-fc=ftn --with-mpi-shared=false --with-debugging=0 --with-fortran-kernels=generic --known-mpi-shared=0 --with-single-library --with-mpi-lib=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/libmpich.a --with-mpi-include=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include --with-debugging=1 --LDFLAGS="-Wl,-z,muldefs" --FFLAGS="-Wl,-z,muldefs" The configure fails with the log info as: ... ... Using built-in specs. COLLECT_GCC=/opt/gcc/4.5.2/bin/../snos/bin/gfortran COLLECT_LTO_WRAPPER=/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/lto-wrapper Target: x86_64-suse-linux Configured with: ../xt-gcc-4.5.2/configure --prefix=/opt/gcc/4.5.2/snos --disable-nls --libdir=/opt/gcc/4.5.2/snos/lib --enable-languages=c,c++,fortran --with-gxx-include-dir=/opt/gcc/4.5.2/snos/include/g++ --with-slibdir=/opt/gcc/4.5.2/snos/lib --with-system-zlib --enable-shared --enable-__cxa_atexit x86_64-suse-linux --with-mpc=/opt/gcc/mpc/0.8.1 --with-mpfr=/opt/gcc/mpfr/2.4.2 --with-gmp=/opt/gcc/gmp/4.3.2 --with-sysroot= Thread model: posix gcc version 4.5.2 20101216 (Cray Inc.) (GCC) COMPILER_PATH=/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/ LIBRARY_PATH=/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS='-u' 'pthread_mutex_trylock' '-fno-second-underscore' '-static' '-o' 'conftest' '-v' '-g' '-L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64' '-L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64' '-L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64' '-L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64' '-L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64' '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45' '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib' '-L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64' '-L/opt/xt-libsci/10.5.0/gnu/lib/45' '-L/opt/xt-libsci/10.5.0/gnu/lib' '-L/opt/cray/portals/default/lib64' '-L/usr/lib/alps' '-L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2' '-L/opt/gcc/4.5.2/snos/lib64' '-L/opt/gcc/4.5.2/snos/lib' '-L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64' '-L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64' '-L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64' '-L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64' '-L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64' '-I/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/include' '-I/opt/cray/gni-headers/2.1-1.0301.2792.5.1.gem/include' '-I/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/include' '-I/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/include' '-I/opt/cray/gni-headers/2.1-1.0301.2792.5.1.gem/include' '-I/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/include' '-I/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/include' '-D__x86_64__' '-D__CRAYXE' '-D__CRAYXT_COMPUTE_LINUX_TARGET' '-D__TARGET_LINUX__' '-I/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include/45' '-I/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include' '-I/opt/cray/mpt/5.1.4/xt/gemini/sma/include' '-I/opt/xt-libsci/10.5.0/gnu/lib/45' '-I/opt/xt-libsci/10.5.0/gnu/lib' '-I/opt/xt-libsci/10.5.0/gnu/include' '-I/usr/include/alps' '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45' '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib' '-L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64' '-L/opt/xt-libsci/10.5.0/gnu/lib/45' '-L/opt/xt-libsci/10.5.0/gnu/lib' '-L/usr/lib/alps' '-mtune=generic' '-march=x86-64' /opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/collect2 --sysroot= -m elf_x86_64 -static -o conftest -u pthread_mutex_trylock /usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o /opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/crtbeginT.o -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 -L/opt/xt-libsci/10.5.0/gnu/lib -L/opt/cray/portals/default/lib64 -L/usr/lib/alps -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 -L/opt/gcc/4.5.2/snos/lib64 -L/opt/gcc/4.5.2/snos/lib -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 -L/opt/xt-libsci/10.5.0/gnu/lib -L/usr/lib/alps -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../.. -z muldefs conftest.o -ldl -lgfortran -lm -lsci -lmpich -lrt -lsma -lxpmem -ldmapp -lugni -lpmi -lalpslli -lalpsutil -ludreg -lpthread -lgomp -lgcc_eh -ldl -rpath=/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 -rpath=/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 -rpath=/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 -rpath=/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 -rpath=/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 -rpath=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 -rpath=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib -rpath=/opt/cray/mpt/default/xt/gemini/mpich2-gnu/lib/45 -rpath=/opt/cray/mpt/default/xt/gemini/mpich2-gnu/lib -rpath=/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -rpath=/opt/cray/mpt/default/xt/gemini/sma/lib64 -rpath=/opt/xt-libsci/10.5.0/gnu/lib/45 -rpath=/opt/xt-libsci/default/gnu/lib/45 -rpath=/opt/xt-libsci/10.5.0/gnu/lib -rpath=/opt/xt-libsci/default/gnu/lib/45 -rpath=/usr/lib/alps --start-group -lsci -lgfortran -lm -lmpichf90 -lmpich -lrt -lsma -lxpmem -ldmapp -lugni -lpmi -lalpslli -lalpsutil -ludreg --whole-archive -lpthread --no-whole-archive --end-group -lgomp -lpthread -lgfortran -lm --start-group -lgcc -lgcc_eh -lc --end-group /opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/crtend.o /usr/lib/../lib64/crtn.o /usr/lib/../lib64/libpthread.a(sem_open.o): In function `sem_open': /usr/src/packages/BUILD/glibc-2.9/nptl/sem_open.c:330: warning: the use of `mktemp' is dangerous, better use `mkstemp' } Pushing language FC Popping language FC in ftn -o conftest -v -Wl,-z,muldefs -g conftest.o -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 -L/opt/xt-libsci/10.5.0/gnu/lib -L/opt/cray/portals/default/lib64 -L/usr/lib/alps -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 -L/opt/gcc/4.5.2/snos/lib64 -L/opt/gcc/4.5.2/snos/lib -ldl -lgfortran -lsci -lmpich -lrt -lsma -lxpmem -ldmapp -lugni -lpmi -lalpslli -lalpsutil -ludreg -lpthread -lgomp -lgcc_eh -ldl Source: program main end Popping language FC ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Mismatched single quotes in Fortran library string ******************************************************************************* File "./configure", line 257, in petsc_configure framework.configure(out = sys.stdout) File "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/framework.py", line 944, in configure child.configure() File "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/compilers.py", line 1220, in configure self.executeTest(self.checkFortranLibraries) File "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/base.py", line 97, in executeTest ret = apply(test, args,kargs) File "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/compilers.py", line 680, in checkFortranLibraries if output.count('\'')%2: raise RuntimeError('Mismatched single quotes in Fortran library string') I'd like to know what causes the mismatched single quotes error. However the same configure options work fine with petsc-3.0.0-p12 without any error. Thanks, Yudong ________________________________________________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. ________________________________________________________________________ From jchludzinski at gmail.com Sun Jul 10 12:45:59 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Sun, 10 Jul 2011 13:45:59 -0400 Subject: [petsc-users] [petsc-maint #78983] Re: MPI v. Pthreads ? In-Reply-To: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: You mentioned: "pthread based Vec and Mat class that lives inside the PETSc MPI world". Is this pthreads *in lieu of* MPI or is it pthreads *in addition to* MPI? ---John On Sun, Jul 10, 2011 at 1:19 PM, Barry Smith wrote: > > On Jul 9, 2011, at 11:57 PM, John Chludzinski wrote: > > > 1) Since Linux now (as of the 2.6 kernel) supports a 1-to-1 > > association/mapping between pthreads (user threads) and kthreads (kernel > > threads), plus the new process scheduler, this makes pthreads much more > > efficient/useful for MIMD parallelism. I assume this makes a > thread-based > > solution (PETSc) more attractive? When PETSc was first conceived, this > was > > not the case. > > > > 2) Since threads can share data (variables), wouldn't there be less of a > > need to send/receive data than using MPI, which simply spawns separate > > disjoint processes? > > > > 3) " Vec and Mat class"? I hope you're sticking with C (v. C++)? I've > > used G++ version 4.3.4 and version 4.5.2 and got different answers with > the > > exact same source. I have used Visual C++ (2008) and got different > answers > > than G++ 4.6.1 - again the same source. C99 seems to give consistent > > results across compilers/platforms and is simpler to boot. > > We call them "classes" because they provide encapsulation, > polymorphism, and inheritance but yes it will remain C. > > Barry > > > > > ---John > > > > > > On Sat, Jul 9, 2011 at 11:41 PM, Barry Smith wrote: > > > >> > >> We have started a pthread based Vec and Mat class that lives inside the > >> PETSc MPI world. We will be pushing it into the petsc-dev repository in > the > >> next could of weeks. > >> > >> Notes: > >> > >> 1) Since all the numerical computations take place in the Vec and Mat > class > >> those are the ones that need to use pthreads underneath, the other > classes > >> like KSP do not. > >> 2) A group in England has developed a OpenMP based Vec and Mat class in > the > >> same vain as our pthread version; I've been trying without success to > get it > >> also into petsc-dev, hopefully I'll eventually charm them into it. > >> 3) Since sparse iterative solvers are very much memory bandwidth limited > >> (the time to solve is determined by the speed of the memory, not the > CPUs) > >> there is only some much one can do to improve performance by throwing > more > >> cores (threads) at the problem. For example most cheap desktop systems > do > >> not have enough memory bandwidth to support even two cores doing the > >> iterative solver so adding pthreads will only matter on certain (more > >> expensive) systems and is not a "cure all". See > >> http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers > >> > >> > >> Barry > >> > >> > >> On Jul 9, 2011, at 10:31 PM, John Chludzinski wrote: > >> > >>> Have you guys considered a pthread based implementation of PETSc/SLEPc. > >>> Have you investigated the use of pthreads? ---John > >>> > >> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Sun Jul 10 12:47:51 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sun, 10 Jul 2011 12:47:51 -0500 (CDT) Subject: [petsc-users] petsc-3.1-p8 configure error In-Reply-To: <4E17111C.6000502@nag.co.uk> References: <4E17111C.6000502@nag.co.uk> Message-ID: On cray - suggest using --with-clib-autodetect=0 --with-cxxlib-autodetect=0 --with-fortranlib-autodetect=0 Check config/examples/arch-cray-xt5-opt.py Satish On Fri, 8 Jul 2011, Yudong Sun wrote: > I am configuring petsc-3.1-p8 for GNU compilers with the options as follows. > > ./configure --known-mpi-c-double-complex=0 --known-level1-dcache-size=65536 > --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=2 > --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 > --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 > --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 > --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=4 > --known-sizeof-MPI_Fint=8 --known-mpi-long-double=0 --with-x=0 --with-batch > --with-blas-lib=-lsci --with-lapack-lib=-lsci --with-dynamic=false > --with-shared=false --CXX=CC --CC=cc --CPP=cpp --with-cpp=cpp --FC=ftn > --with-fc=ftn --with-mpi-shared=false --with-debugging=0 > --with-fortran-kernels=generic --known-mpi-shared=0 --with-single-library > --with-mpi-lib=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/libmpich.a > --with-mpi-include=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include > --with-debugging=1 --LDFLAGS="-Wl,-z,muldefs" --FFLAGS="-Wl,-z,muldefs" > > > > The configure fails with the log info as: > > ... ... > Using built-in specs. > COLLECT_GCC=/opt/gcc/4.5.2/bin/../snos/bin/gfortran > COLLECT_LTO_WRAPPER=/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/lto-wrapper > Target: x86_64-suse-linux > Configured with: ../xt-gcc-4.5.2/configure --prefix=/opt/gcc/4.5.2/snos > --disable-nls --libdir=/opt/gcc/4.5.2/snos/lib > --enable-languages=c,c++,fortran > --with-gxx-include-dir=/opt/gcc/4.5.2/snos/include/g++ > --with-slibdir=/opt/gcc/4.5.2/snos/lib --with-system-zlib --enable-shared > --enable-__cxa_atexit x86_64-suse-linux --with-mpc=/opt/gcc/mpc/0.8.1 > --with-mpfr=/opt/gcc/mpfr/2.4.2 --with-gmp=/opt/gcc/gmp/4.3.2 --with-sysroot= > Thread model: posix > gcc version 4.5.2 20101216 (Cray Inc.) (GCC) > COMPILER_PATH=/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/ > LIBRARY_PATH=/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../:/lib/:/usr/lib/ > COLLECT_GCC_OPTIONS='-u' 'pthread_mutex_trylock' '-fno-second-underscore' > '-static' '-o' 'conftest' '-v' '-g' > '-L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64' > '-L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64' > '-L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64' > '-L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64' > '-L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64' > '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45' > '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib' > '-L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64' > '-L/opt/xt-libsci/10.5.0/gnu/lib/45' '-L/opt/xt-libsci/10.5.0/gnu/lib' > '-L/opt/cray/portals/default/lib64' '-L/usr/lib/alps' > '-L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2' > '-L/opt/gcc/4.5.2/snos/lib64' '-L/opt/gcc/4.5.2/snos/lib' > '-L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64' > '-L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64' > '-L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64' > '-L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64' > '-L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64' > '-I/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/include' > '-I/opt/cray/gni-headers/2.1-1.0301.2792.5.1.gem/include' > '-I/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/include' > '-I/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/include' > '-I/opt/cray/gni-headers/2.1-1.0301.2792.5.1.gem/include' > '-I/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/include' > '-I/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/include' '-D__x86_64__' '-D__CRAYXE' > '-D__CRAYXT_COMPUTE_LINUX_TARGET' '-D__TARGET_LINUX__' > '-I/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include/45' > '-I/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/include' > '-I/opt/cray/mpt/5.1.4/xt/gemini/sma/include' > '-I/opt/xt-libsci/10.5.0/gnu/lib/45' '-I/opt/xt-libsci/10.5.0/gnu/lib' > '-I/opt/xt-libsci/10.5.0/gnu/include' '-I/usr/include/alps' > '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45' > '-L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib' > '-L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64' > '-L/opt/xt-libsci/10.5.0/gnu/lib/45' '-L/opt/xt-libsci/10.5.0/gnu/lib' > '-L/usr/lib/alps' '-mtune=generic' '-march=x86-64' > /opt/gcc/4.5.2/snos/libexec/gcc/x86_64-suse-linux/4.5.2/collect2 --sysroot= > -m elf_x86_64 -static -o conftest -u pthread_mutex_trylock > /usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o > /opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/crtbeginT.o > -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 > -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 > -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 > -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 > -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib > -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 > -L/opt/xt-libsci/10.5.0/gnu/lib -L/opt/cray/portals/default/lib64 > -L/usr/lib/alps -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 > -L/opt/gcc/4.5.2/snos/lib64 -L/opt/gcc/4.5.2/snos/lib > -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 > -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 > -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 > -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 > -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib > -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 > -L/opt/xt-libsci/10.5.0/gnu/lib -L/usr/lib/alps > -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 > -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../../../lib64 > -L/lib/../lib64 -L/usr/lib/../lib64 > -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/../../.. -z muldefs > conftest.o -ldl -lgfortran -lm -lsci -lmpich -lrt -lsma -lxpmem -ldmapp -lugni > -lpmi -lalpslli -lalpsutil -ludreg -lpthread -lgomp -lgcc_eh -ldl > -rpath=/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 > -rpath=/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 > -rpath=/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 > -rpath=/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 > -rpath=/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 > -rpath=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 > -rpath=/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib > -rpath=/opt/cray/mpt/default/xt/gemini/mpich2-gnu/lib/45 > -rpath=/opt/cray/mpt/default/xt/gemini/mpich2-gnu/lib > -rpath=/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 > -rpath=/opt/cray/mpt/default/xt/gemini/sma/lib64 > -rpath=/opt/xt-libsci/10.5.0/gnu/lib/45 > -rpath=/opt/xt-libsci/default/gnu/lib/45 -rpath=/opt/xt-libsci/10.5.0/gnu/lib > -rpath=/opt/xt-libsci/default/gnu/lib/45 -rpath=/usr/lib/alps --start-group > -lsci -lgfortran -lm -lmpichf90 -lmpich -lrt -lsma -lxpmem -ldmapp -lugni > -lpmi -lalpslli -lalpsutil -ludreg --whole-archive -lpthread > --no-whole-archive --end-group -lgomp -lpthread -lgfortran -lm --start-group > -lgcc -lgcc_eh -lc --end-group > /opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2/crtend.o > /usr/lib/../lib64/crtn.o > /usr/lib/../lib64/libpthread.a(sem_open.o): In function `sem_open': > /usr/src/packages/BUILD/glibc-2.9/nptl/sem_open.c:330: warning: the use of > `mktemp' is dangerous, better use `mkstemp' > } > Pushing language FC > Popping language FC > in ftn -o conftest -v -Wl,-z,muldefs -g conftest.o > -L/opt/cray/udreg/2.1-1.0301.2797.5.2.gem/lib64 > -L/opt/cray/ugni/2.1-1.0301.2864.7.44.gem/lib64 > -L/opt/cray/dmapp/2.2-1.0301.2791.5.1.gem/lib64 > -L/opt/cray/xpmem/0.1-2.0301.24575.5.2.gem/lib64 > -L/opt/cray/pmi/1.0-1.0000.8160.39.2.gem/lib64 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib/45 > -L/opt/cray/mpt/5.1.4/xt/gemini/mpich2-gnu/lib > -L/opt/cray/mpt/5.1.4/xt/gemini/sma/lib64 -L/opt/xt-libsci/10.5.0/gnu/lib/45 > -L/opt/xt-libsci/10.5.0/gnu/lib -L/opt/cray/portals/default/lib64 > -L/usr/lib/alps -L/opt/gcc/4.5.2/snos/lib/gcc/x86_64-suse-linux/4.5.2 > -L/opt/gcc/4.5.2/snos/lib64 -L/opt/gcc/4.5.2/snos/lib -ldl -lgfortran -lsci > -lmpich -lrt -lsma -lxpmem -ldmapp -lugni -lpmi -lalpslli -lalpsutil -ludreg > -lpthread -lgomp -lgcc_eh -ldl > Source: > program main > > end > Popping language FC > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > ------------------------------------------------------------------------------- > Mismatched single quotes in Fortran library string > ******************************************************************************* > File "./configure", line 257, in petsc_configure > framework.configure(out = sys.stdout) > File > "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/framework.py", > line 944, in configure > child.configure() > File > "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/compilers.py", > line 1220, in configure > self.executeTest(self.checkFortranLibraries) > File > "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/base.py", > line 97, in executeTest > ret = apply(test, args,kargs) > File > "/esfs2/z03/z03/ydsun/queries/q173396_petsc/petsc-3.1-p8/config/BuildSystem/config/compilers.py", > line 680, in checkFortranLibraries > if output.count('\'')%2: raise RuntimeError('Mismatched single quotes in > Fortran library string') > > > I'd like to know what causes the mismatched single quotes error. However the > same configure options work fine with petsc-3.0.0-p12 without any error. > > Thanks, > > Yudong > > ________________________________________________________________________ > The Numerical Algorithms Group Ltd is a company registered in England > and Wales with company number 1249803. The registered office is: > Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. > > This e-mail has been scanned for all viruses by Star. The service is > powered by MessageLabs. > ________________________________________________________________________ > From jedbrown at mcs.anl.gov Sun Jul 10 12:48:08 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 12:48:08 -0500 Subject: [petsc-users] [petsc-maint #78983] Re: MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: On Sun, Jul 10, 2011 at 12:45, John Chludzinski wrote: > You mentioned: "pthread based Vec and Mat class that lives inside the PETSc > MPI world". Is this pthreads *in lieu of* MPI or is it pthreads *in > addition to* MPI? > The latter. Pthreads within a NUMA node or shared memory node, MPI between nodes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Sun Jul 10 12:57:12 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Sun, 10 Jul 2011 13:57:12 -0400 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: So if host the "new and improved" PETSc on a mult--proc/multi-core Linux workstation (or in my case, a mult--proc/multi-core Windows workstation running Cygwin), not configured as a cluster, I would be using pthreads and not MPI? ---John On Sun, Jul 10, 2011 at 12:45, John Chludzinski wrote: > You mentioned: "pthread based Vec and Mat class that lives inside the PETSc > MPI world". Is this pthreads *in lieu of* MPI or is it pthreads *in > addition to* MPI? > The latter. Pthreads within a NUMA node or shared memory node, MPI between nodes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 10 13:00:19 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 13:00:19 -0500 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: On Sun, Jul 10, 2011 at 12:57, John Chludzinski wrote: > So if host the "new and improved" PETSc on a mult--proc/multi-core Linux > workstation (or in my case, a mult--proc/multi-core Windows workstation > running Cygwin), not configured as a cluster, I would be using pthreads and > not MPI? If it is a single machine, you would be able to use any combination from all separate MPI processes to one process. MPI calls won't go away, all the XXCreate() functions will still have the MPI_Comm argument. You'll be able to use MPIUNI if you don't have a real MPI installed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Sun Jul 10 13:15:21 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Sun, 10 Jul 2011 14:15:21 -0400 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: "If you don't have a real MPI installed"? I installed (on Cygwin) using: ./configure CC=gcc FC=gfortran --download-mpich=1 PETSC_ARCH=arch-cygwin-gnu I've compiled some example MPI code using mpicc. And I've run the generated executable with: mpiexec -n . "ps" says it created n-number of processes. But it is on a 2-proc/4-core Windows box running Cygwin (of course, not configured as a cluster). Do I have "real MPI" installed? ---John On Sun, Jul 10, 2011 at 2:00 PM, Jed Brown wrote: > On Sun, Jul 10, 2011 at 12:57, John Chludzinski wrote: > >> So if host the "new and improved" PETSc on a mult--proc/multi-core Linux >> workstation (or in my case, a mult--proc/multi-core Windows workstation >> running Cygwin), not configured as a cluster, I would be using pthreads and >> not MPI? > > > If it is a single machine, you would be able to use any combination from > all separate MPI processes to one process. MPI calls won't go away, all the > XXCreate() functions will still have the MPI_Comm argument. You'll be able > to use MPIUNI if you don't have a real MPI installed. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 10 13:23:19 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 13:23:19 -0500 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: On Sun, Jul 10, 2011 at 13:15, John Chludzinski wrote: > "If you don't have a real MPI installed"? > > I installed (on Cygwin) using: ./configure CC=gcc FC=gfortran > --download-mpich=1 PETSC_ARCH=arch-cygwin-gnu > > I've compiled some example MPI code using mpicc. And I've run the generated > executable with: mpiexec -n . > > "ps" says it created n-number of processes. > > But it is on a 2-proc/4-core Windows box running Cygwin (of course, not > configured as a cluster). > > Do I have "real MPI" installed? > Yes. Note that threading offers very little potential performance improvement on a system like this. It becomes more important if you have many "fat" nodes, for example if each node has 4 sockets with 12 cores per socket and you want to "strong scale" such that the subdomains become very small (less than 10k unknowns if inexpensive preconditioners are working, also depending on the network and intra-node bandwidth). -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Sun Jul 10 13:36:45 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Sun, 10 Jul 2011 14:36:45 -0400 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: I assume you mean: neither pthreads nor MPI "offers very little potential performance improvement on a system like this"? I work for a small company (in Huntsville) of mechanical engineers with a limited budget where such machines are their staple. I'm looking at PETSc and SLEPc to help with some of their large scale problems (e.g., generalized eigenvalue problems for matrices from 10K x 10K to matrices over 100K x 100K). I'm also looking at ways to parallelize of own codes, hence pthreads vs. MPI? ---John On Sun, Jul 10, 2011 at 2:23 PM, Jed Brown wrote: > On Sun, Jul 10, 2011 at 13:15, John Chludzinski wrote: > >> "If you don't have a real MPI installed"? >> >> I installed (on Cygwin) using: ./configure CC=gcc FC=gfortran >> --download-mpich=1 PETSC_ARCH=arch-cygwin-gnu >> >> I've compiled some example MPI code using mpicc. And I've run the >> generated executable with: mpiexec -n . >> >> "ps" says it created n-number of processes. >> >> But it is on a 2-proc/4-core Windows box running Cygwin (of course, not >> configured as a cluster). >> >> Do I have "real MPI" installed? >> > > Yes. > > Note that threading offers very little potential performance improvement on > a system like this. It becomes more important if you have many "fat" nodes, > for example if each node has 4 sockets with 12 cores per socket and you want > to "strong scale" such that the subdomains become very small (less than 10k > unknowns if inexpensive preconditioners are working, also depending on the > network and intra-node bandwidth). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 10 13:52:27 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 13:52:27 -0500 Subject: [petsc-users] MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: On Sun, Jul 10, 2011 at 13:36, John Chludzinski wrote: > I assume you mean: neither pthreads nor MPI "offers very little potential > performance improvement on a system like this"? > No, you can get a decent parallel speedup with smaller systems or "thin" nodes. It should be at least as much as the increase in memory bandwidth, and can be better if your workload has enough floating point operations. My claim was that the difference between all-MPI and hybrid MPI/pthreads will not be large on this sort of hardware. So you don't suffer from just using MPI everywhere. If the heavy computation is well-isolated to either a library or a localized bit of code, then it's not difficult to add threading later (e.g. when you switch hardware). > > I work for a small company (in Huntsville) of mechanical engineers with a > limited budget where such machines are their staple. I'm looking at PETSc > and SLEPc to help with some of their large scale problems (e.g., generalized > eigenvalue problems for matrices from 10K x 10K to matrices over 100K x > 100K). > This problem size is not very large. > I'm also looking at ways to parallelize of own codes, hence pthreads vs. > MPI? > MPI forces you to think about data locality up-front, which almost always produces better software, even if you end up also using threads. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Sun Jul 10 15:03:00 2011 From: zonexo at gmail.com (TAY wee-beng) Date: Sun, 10 Jul 2011 22:03:00 +0200 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> <4E18D9AE.3040106@gmail.com> <4E18DCEF.6020107@gmail.com> Message-ID: <4E1A0574.4060505@gmail.com> Hi Jed, I think I understand what you mean. Are you saying that I have 2 separate nxn matrix to solve. I combine them to form a 2n x 2n matrix and solve. I think it will also improve the scalability with regards to MPI too. Is that so? Yours sincerely, TAY wee-beng On 10/7/2011 1:02 AM, Jed Brown wrote: > On Sat, Jul 9, 2011 at 17:57, TAY wee-beng > wrote: > > Ok thanks. Btw, is it better to solve my x/y eqns together? How > can I do that? > > > Sure, interlace the x and y degrees of freedom and assemble the matrix > with x equations in the even row/column numbers and y equations in the > odd row/column numbers. > > Note that the equations are not decoupled for some boundary conditions > (e.g. slip on a curved surface or free surface). -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 10 15:04:03 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 10 Jul 2011 15:04:03 -0500 Subject: [petsc-users] Petsc has generated inconsistent data - Divide by zero! In-Reply-To: <4E1A0574.4060505@gmail.com> References: <4E16F7A4.3060200@gmail.com> <4E170578.2030209@gmail.com> <4E1781DE.7070207@gmail.com> <4E18D47D.8060807@gmail.com> <4E18D9AE.3040106@gmail.com> <4E18DCEF.6020107@gmail.com> <4E1A0574.4060505@gmail.com> Message-ID: On Sun, Jul 10, 2011 at 15:03, TAY wee-beng wrote: > I think I understand what you mean. Are you saying that I have 2 separate > nxn matrix to solve. > > I combine them to form a 2n x 2n matrix and solve. I think it will also > improve the scalability with regards to MPI too. > Yes -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jul 10 21:20:29 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jul 2011 21:20:29 -0500 Subject: [petsc-users] [petsc-maint #78993] Re: MPI v. Pthreads ? In-Reply-To: References: <50CB7EBF-5318-4BAB-97A8-BD572B9EFFBB@mcs.anl.gov> Message-ID: <5692BD5C-3255-419C-B185-07DA5B3A725F@mcs.anl.gov> On Jul 10, 2011, at 1:02 PM, John Chludzinski wrote: > So if host the "new and improved" PETSc on a mult--proc/multi-core Linux > workstation (or in my case, a mult--proc/multi-core Windows workstation > running Cygwin), not configured as a cluster, I would be using pthreads and > not MPI? You can mix and match. One MPI process and a bunch of threads or two (or more) MPI processes each using some threads. Barry > > ---John > > > On Sun, Jul 10, 2011 at 12:45, John Chludzinski wrote: > >> You mentioned: "pthread based Vec and Mat class that lives inside the PETSc >> MPI world". Is this pthreads *in lieu of* MPI or is it pthreads *in >> addition to* MPI? >> > > The latter. Pthreads within a NUMA node or shared memory node, MPI between > nodes. > From Kevin.Green at uoit.ca Mon Jul 11 18:28:21 2011 From: Kevin.Green at uoit.ca (Kevin Green) Date: Mon, 11 Jul 2011 19:28:21 -0400 Subject: [petsc-users] Field dependent stencil in DAs Message-ID: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> Greetings, I have been playing around with PetSc for the past couple of weeks, and have a question to which I have not found an answer in either the examples or the mailing list archives. Essentially, I'm (currently) looking to numerically solve a system of 5 time dependent equations, but only 1 of the fields is involved with spatial derivatives. Now, I've found a couple of ways to do this, either by introducing a Field struct cf. ${PETSC_DIR}/src/snes/examples/tutorials/ex19.c, or by changing one of the various ts examples to use DAVecGetArrayDOF(...). Would there be any performance difference between introducing a Field struct compared to DAVecGetArrayDOF(...)? Intuitively, I would think not...but I suppose I should ask since I'm here anyway. Now the real reason I came: Is there some sort of mask that can be applied to the DAs that would turn off the communication of ghost points for some degrees of freedom? I assume that the communication structures are set up so that only a single message needs to be passed between communicating processes on update, but this still results in passing 5X the data that I have to. Any comment on this would be much appreciated. Thank you, Kevin From ecoon at lanl.gov Mon Jul 11 18:36:06 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Mon, 11 Jul 2011 17:36:06 -0600 Subject: [petsc-users] Field dependent stencil in DAs In-Reply-To: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> References: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> Message-ID: <1310427366.24206.122.camel@echo.lanl.gov> On Mon, 2011-07-11 at 19:28 -0400, Kevin Green wrote: > Now the real reason I came: Is there some sort of mask that can be applied to the DAs that would turn off the communication of ghost points for some degrees of freedom? No. However, it would be easy enough to create a 1-dof DA of the same (local and global) sizes as your 5-dof DA, create a Vec using that dof, scatter between the 5-dof Vec and the 1-dof Vec (see VecStrideGather()), then communicate only this 1-dof Vec. Ethan > I assume that the communication structures are set up so that only a single message needs to be passed between communicating processes on update, but this still results in passing 5X the data that I have to. Any comment on this would be much appreciated. > > Thank you, > Kevin -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From jedbrown at mcs.anl.gov Mon Jul 11 18:38:16 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 11 Jul 2011 18:38:16 -0500 Subject: [petsc-users] Field dependent stencil in DAs In-Reply-To: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> References: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> Message-ID: On Mon, Jul 11, 2011 at 18:28, Kevin Green wrote: > Essentially, I'm (currently) looking to numerically solve a system of 5 > time dependent equations, but only 1 of the fields is involved with spatial > derivatives. Now, I've found a couple of ways to do this, either by > introducing a Field struct cf. > ${PETSC_DIR}/src/snes/examples/tutorials/ex19.c, or by changing one of the > various ts examples to use DAVecGetArrayDOF(...). Would there be any > performance difference between introducing a Field struct compared to > DAVecGetArrayDOF(...)? Intuitively, I would think not...but I suppose I > should ask since I'm here anyway. > I like using field structs because then I get to use names instead of numbers for my fields and because there is one less level of pointer indirection. > > Now the real reason I came: Is there some sort of mask that can be applied > to the DAs that would turn off the communication of ghost points for some > degrees of freedom? I assume that the communication structures are set up > so that only a single message needs to be passed between communicating > processes on update, but this still results in passing 5X the data that I > have to. Any comment on this would be much appreciated. > DMDASetBlockFills() -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Tue Jul 12 03:47:00 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Tue, 12 Jul 2011 10:47:00 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> Message-ID: Jed, thanks for your answer. This is exactly what I was looking for (it took me some time to test it...). Let me ask you a couple of additional questions: 1) in this configuration, is there an alternative way to pass the ordering to the preconditioner? Becuse my ordering depends on the grid an not on the solution, it is constant during all the computations and I have no advantage on calling the function that creates multiple times (the pc is built and destroyed several times). 2) what's the best way to make the shell pc selectable from the option database, so that it can be replaced with more standard ones? Thanks Gianluca On 1 July 2011 14:43, Jed Brown wrote: > On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello > wrote: >> >> I've tried to implement Jed's solution as it was looking faster to >> try. This is what I came up with for two different orders, FENOrder >> and FWNOrder respectively >> >> 283 ? ? ? KSP ksp; >> 284 ? ? ? PC pc; >> 285 >> 286 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >> 287 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >> 288 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >> CHKERRQ(ierr); >> 289 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >> 290 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >> 291 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >> 292 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >> 293 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >> 294 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); >> 295 >> 296 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >> 297 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >> 298 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >> CHKERRQ(ierr); >> 299 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >> 300 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >> 301 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >> 302 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >> 303 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >> 304 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); > > It looks like you are putting all of this in PCApply_YourShell(). I suggest > making > struct YourShellContext { > ? PC fenpc,fwnpc; > ? Vec work; > ? PetscBool issetup; > }; > Then pass a pointer to this thing in when you create the PCShell. Then > something like > PCSetUp_YourShell(PC pc) { > struct YourShellContext *ctx; > Mat A,B; > MatStructure mstr; > const char *prefix; > char iprefix[256]; > PCShellGetContext(pc,&ctx); > PCGetOptionsPrefix(pc,&prefix); > if (!ctx->issetup) { > ? ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); > ? PCCreate(comm,&ctx->fenpc); > ? PCSetType(ctx->fenpc,PCILU); > ? PCFactorSetFill(ctx->fenpc,1); > ? PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); > ? snprintf(iprefix,sizeof iprefix,"%sfen",prefix); > ? PCSetOptionsPrefix(ctx->fenpc,iprefix); > ? PCSetFromOptions(ctx->fenpc); > ? /* Same as above for ctx->fwnpc */ > } > PCGetOperators(pc,&A,&B,&mstr); > PCSetOperators(ctx->fenpc,A,B,mstr); > PCSetOperators(ctx->fwnpc,A,B,mstr); > PCSetUp(ctx->fenpc); > PCSetUp(ctx->fwnpc); > ctx->issetup = PETSC_TRUE; > } > Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { > ... > PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); > PCApply(ctx->fenpc,X,Y); /* apply first ordering */ > VecScale(Y,-1.0); > MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ > PCApply(ctx->fwnpc,ctx->work,Y); > } >> >> I don't know if this is what you intended... is there a way to make >> KSP recompute the factorization with the different ordering without >> destroying and recreating the KSP? > > Store both as in the code above. > >> >> Concerning Barry's option, js it possible to use the PETSc ilu >> factorization function (and eventually the ones provided by external >> libraries) inside my PCSHELL, passing it the various ordering I will >> need? If so, what's the best way to access it? > > Does the code above answer this question? -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From bsmith at mcs.anl.gov Tue Jul 12 13:15:11 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 12 Jul 2011 13:15:11 -0500 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> Message-ID: <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: > Jed, thanks for your answer. > > This is exactly what I was looking for (it took me some time to test it...). > > Let me ask you a couple of additional questions: > > 1) in this configuration, is there an alternative way to pass the > ordering to the preconditioner? Becuse my ordering depends on the grid > an not on the solution, it is constant during all the computations and > I have no advantage on calling the function that creates multiple > times (the pc is built and destroyed several times). Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. PCShellSet/GetContext()? > > 2) what's the best way to make the shell pc selectable from the option > database, so that it can be replaced with more standard ones? > /* here the user can provide -pc_type shell or ilu or whatever */ KSPSetFromOptions(ksp); KSPGetPC(ksp,&pc); /* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ PCShellSetContext(pc,....) PCShellSetSetUp(pc,....) PCShellSetApply(pc,.....) Barry > Thanks > > Gianluca > > On 1 July 2011 14:43, Jed Brown wrote: >> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >> wrote: >>> >>> I've tried to implement Jed's solution as it was looking faster to >>> try. This is what I came up with for two different orders, FENOrder >>> and FWNOrder respectively >>> >>> 283 KSP ksp; >>> 284 PC pc; >>> 285 >>> 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>> 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>> 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>> CHKERRQ(ierr); >>> 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>> 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>> 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>> 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>> 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>> 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>> 295 >>> 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>> 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>> 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>> CHKERRQ(ierr); >>> 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>> 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>> 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>> 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>> 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>> 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >> >> It looks like you are putting all of this in PCApply_YourShell(). I suggest >> making >> struct YourShellContext { >> PC fenpc,fwnpc; >> Vec work; >> PetscBool issetup; >> }; >> Then pass a pointer to this thing in when you create the PCShell. Then >> something like >> PCSetUp_YourShell(PC pc) { >> struct YourShellContext *ctx; >> Mat A,B; >> MatStructure mstr; >> const char *prefix; >> char iprefix[256]; >> PCShellGetContext(pc,&ctx); >> PCGetOptionsPrefix(pc,&prefix); >> if (!ctx->issetup) { >> ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >> PCCreate(comm,&ctx->fenpc); >> PCSetType(ctx->fenpc,PCILU); >> PCFactorSetFill(ctx->fenpc,1); >> PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >> snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >> PCSetOptionsPrefix(ctx->fenpc,iprefix); >> PCSetFromOptions(ctx->fenpc); >> /* Same as above for ctx->fwnpc */ >> } >> PCGetOperators(pc,&A,&B,&mstr); >> PCSetOperators(ctx->fenpc,A,B,mstr); >> PCSetOperators(ctx->fwnpc,A,B,mstr); >> PCSetUp(ctx->fenpc); >> PCSetUp(ctx->fwnpc); >> ctx->issetup = PETSC_TRUE; >> } >> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >> ... >> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >> VecScale(Y,-1.0); >> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >> PCApply(ctx->fwnpc,ctx->work,Y); >> } >>> >>> I don't know if this is what you intended... is there a way to make >>> KSP recompute the factorization with the different ordering without >>> destroying and recreating the KSP? >> >> Store both as in the code above. >> >>> >>> Concerning Barry's option, js it possible to use the PETSc ilu >>> factorization function (and eventually the ones provided by external >>> libraries) inside my PCSHELL, passing it the various ordering I will >>> need? If so, what's the best way to access it? >> >> Does the code above answer this question? > > > > -- > "[Je pense que] l'homme est un monde qui vaut des fois les mondes et > que les plus ardentes ambitions sont celles qui ont eu l'orgueil de > l'Anonymat" -- Non omnibus, sed mihi et tibi > Amedeo Modigliani From jchludzinski at gmail.com Tue Jul 12 23:41:27 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 13 Jul 2011 00:41:27 -0400 Subject: [petsc-users] Storing a matrix in PETSc binary form? Message-ID: I have a matrix stored as a sequential list of binary values (not PETSc binary form) that I read in with a single block read: read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). How do I convert this simple binary file of (sequentially stored) doubles into a "PETSc binary matrix file"? ---John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Tue Jul 12 23:49:21 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 13 Jul 2011 00:49:21 -0400 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: fread( (void *)K, sizeof(double), SIZE*SIZE, fpK ) On Wed, Jul 13, 2011 at 12:41 AM, John Chludzinski wrote: > I have a matrix stored as a sequential list of binary values (not PETSc > binary form) that I read in with a single block read: > > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). > > How do I convert this simple binary file of (sequentially stored) doubles > into a "PETSc binary matrix file"? > > ---John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Jul 13 00:02:49 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 13 Jul 2011 00:02:49 -0500 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: If it is truly a dense matrix then it isn't clear that you want to store it as a PETSc binary matrix but you can using the following code PetscInt n; PetscScalar *values; allocate values the correct size; read the entries into values then call MatCreateSeqDense(PETSC_COMM_SELF,n,n,values,&A); the call MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); Barry On Jul 12, 2011, at 11:41 PM, John Chludzinski wrote: > I have a matrix stored as a sequential list of binary values (not PETSc binary form) that I read in with a single block read: > > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). > > How do I convert this simple binary file of (sequentially stored) doubles into a "PETSc binary matrix file"? > > ---John From jchludzinski at gmail.com Wed Jul 13 00:21:54 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 13 Jul 2011 01:21:54 -0400 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: They're dense. I'm trying to use a SLEPc generalized eigenvalue example that expects matrices stored in PETSc binary files. ---John On Wed, Jul 13, 2011 at 1:02 AM, Barry Smith wrote: > > If it is truly a dense matrix then it isn't clear that you want to store > it as a PETSc binary matrix but you can using the following code > > PetscInt n; > PetscScalar *values; > > allocate values the correct size; read the entries into values then call > MatCreateSeqDense(PETSC_COMM_SELF,n,n,values,&A); the call > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > Barry > > On Jul 12, 2011, at 11:41 PM, John Chludzinski wrote: > > > I have a matrix stored as a sequential list of binary values (not PETSc > binary form) that I read in with a single block read: > > > > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). > > > > How do I convert this simple binary file of (sequentially stored) doubles > into a "PETSc binary matrix file"? > > > > ---John > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Wed Jul 13 02:24:32 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 13 Jul 2011 03:24:32 -0400 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: Is this what you had in mind (more or less)? ---John //-----------------------------------CONVERTER--------------------------------------------------------------------------------------------------------- static char help[] = "Reads matrix: \n -f : file to load \n\n"; #define SIZE 4002 #include #include #include "petscmat.h" int main(int argc,char **args) { Mat A; char file[PETSC_MAX_PATH_LEN]; PetscErrorCode ierr; PetscScalar *a; PetscTruth flg; PetscInt n=SIZE; int i, j; PetscInitialize(&argc,&args,(char *)0,help); ierr = PetscOptionsGetString(PETSC_NULL,"-f",file,PETSC_MAX_PATH_LEN-1,&flg);CHKERRQ(ierr); if (!flg) SETERRQ(1,"Must indicate binary file with the -f option"); ierr = PetscMalloc(SIZE*SIZE*sizeof(PetscScalar),&a);CHKERRQ(ierr); FILE *fpK; double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); if((fpK = fopen(file, "rb")) == NULL) { printf("Cannot open joe_DOF4002_k_double.bin\n"); exit(1); } fprintf(stderr, "Doubles read for K = %d\n", fread( (void *)K, sizeof(double), SIZE*SIZE, fpK )); for ( i = 0; i < SIZE; i++ ) for ( j = i; j < SIZE; j++ ) a[i+j*SIZE]= *(K+i*SIZE+j); MatCreateSeqDense(PETSC_COMM_SELF, n, n, a, &A); MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); ierr = PetscFinalize();CHKERRQ(ierr); return 0; } On Wed, Jul 13, 2011 at 1:02 AM, Barry Smith wrote: > > If it is truly a dense matrix then it isn't clear that you want to store > it as a PETSc binary matrix but you can using the following code > > PetscInt n; > PetscScalar *values; > > allocate values the correct size; read the entries into values then call > MatCreateSeqDense(PETSC_COMM_SELF,n,n,values,&A); the call > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > Barry > > On Jul 12, 2011, at 11:41 PM, John Chludzinski wrote: > > > I have a matrix stored as a sequential list of binary values (not PETSc > binary form) that I read in with a single block read: > > > > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). > > > > How do I convert this simple binary file of (sequentially stored) doubles > into a "PETSc binary matrix file"? > > > > ---John > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Wed Jul 13 02:40:57 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 13 Jul 2011 03:40:57 -0400 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: $ ls -l -rw-r--r-- 1 John None 192208072 Jul 13 03:31 binaryoutput -rw-r--r-- 1 John None 0 Jul 13 03:31 binaryoutput.info -rw-r--r-- 1 John None 128128032 Jul 13 01:43 DOF4002_k_double.bin -rw-r--r-- 1 John None 128128032 Jul 13 01:43 DOF4002_m_double.bin where DOF4002_k_double.bin and DOF4002_m_double.bin are the file to be converted. What is binaryoutput.info (size=0)? And why is binaryoutput so much fatter than either DOF4002_k_double.bin or DOF4002_m_double.bin? ---John On Wed, Jul 13, 2011 at 3:24 AM, John Chludzinski wrote: > Is this what you had in mind (more or less)? ---John > > > //-----------------------------------CONVERTER--------------------------------------------------------------------------------------------------------- > > static char help[] = "Reads matrix: \n -f : file to load > \n\n"; > > #define SIZE 4002 > > #include > #include > #include "petscmat.h" > > int main(int argc,char **args) > { > Mat A; > char file[PETSC_MAX_PATH_LEN]; > PetscErrorCode ierr; > PetscScalar *a; > PetscTruth flg; > PetscInt n=SIZE; > int i, j; > > PetscInitialize(&argc,&args,(char *)0,help); > > ierr = > PetscOptionsGetString(PETSC_NULL,"-f",file,PETSC_MAX_PATH_LEN-1,&flg);CHKERRQ(ierr); > if (!flg) SETERRQ(1,"Must indicate binary file with the -f option"); > > ierr = PetscMalloc(SIZE*SIZE*sizeof(PetscScalar),&a);CHKERRQ(ierr); > > FILE *fpK; > > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > > if((fpK = fopen(file, "rb")) == NULL) > { > printf("Cannot open joe_DOF4002_k_double.bin\n"); > exit(1); > } > > fprintf(stderr, "Doubles read for K = %d\n", fread( (void *)K, > sizeof(double), SIZE*SIZE, fpK )); > > for ( i = 0; i < SIZE; i++ ) > for ( j = i; j < SIZE; j++ ) a[i+j*SIZE]= *(K+i*SIZE+j); > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, a, &A); > > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > ierr = PetscFinalize();CHKERRQ(ierr); > return 0; > } > > > > On Wed, Jul 13, 2011 at 1:02 AM, Barry Smith wrote: > >> >> If it is truly a dense matrix then it isn't clear that you want to store >> it as a PETSc binary matrix but you can using the following code >> >> PetscInt n; >> PetscScalar *values; >> >> allocate values the correct size; read the entries into values then call >> MatCreateSeqDense(PETSC_COMM_SELF,n,n,values,&A); the call >> MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); >> >> Barry >> >> On Jul 12, 2011, at 11:41 PM, John Chludzinski wrote: >> >> > I have a matrix stored as a sequential list of binary values (not PETSc >> binary form) that I read in with a single block read: >> > >> > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). >> > >> > How do I convert this simple binary file of (sequentially stored) >> doubles into a "PETSc binary matrix file"? >> > >> > ---John >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Wed Jul 13 04:02:05 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Wed, 13 Jul 2011 11:02:05 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> Message-ID: Thanks Barry. What I'm doing at the moment in order to pass the ordering to the PC is PCFactorSetMatOrderingType(PC,"FESOrder"); where the FESOrder(...) function computes the ordering and has been registered in the main function with MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); The same is done for al the different orderings I'm using inside the PCSHELL suggested by Jed. My understanding is that the FESOrder(...) function is evaluated every time the PCApply(...) function is called. If there was a way to store the IS containing the ordering ? or any other array ? into the shell context (this I know how to do that) and the pass the IS directly to the preconditioner instead of using PCFactorSetMatOrderingType would be the best solution for me. Thanks Gianluca On 12 July 2011 20:15, Barry Smith wrote: > > On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: > >> Jed, thanks for your answer. >> >> This is exactly what I was looking for (it took me some time to test it...). >> >> Let me ask you a couple of additional questions: >> >> 1) in this configuration, is there an alternative way to pass the >> ordering to the preconditioner? Becuse my ordering depends on the grid >> an not on the solution, it is constant during all the computations and >> I have no advantage on calling the function that creates multiple >> times (the pc is built and destroyed several times). > > ? Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. > > ? ?PCShellSet/GetContext()? >> >> 2) what's the best way to make the shell pc selectable from the option >> database, so that it can be replaced with more standard ones? >> > > ? ? ? ?/* here the user can provide -pc_type shell or ilu or whatever */ > ? ? ? ?KSPSetFromOptions(ksp); > ? ? ? ?KSPGetPC(ksp,&pc); > ? ? ? ?/* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ > > ? ? ? ?PCShellSetContext(pc,....) > ? ? ? ?PCShellSetSetUp(pc,....) > ? ? ? ?PCShellSetApply(pc,.....) > > > ? Barry > > >> Thanks >> >> Gianluca >> >> On 1 July 2011 14:43, Jed Brown wrote: >>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>> wrote: >>>> >>>> I've tried to implement Jed's solution as it was looking faster to >>>> try. This is what I came up with for two different orders, FENOrder >>>> and FWNOrder respectively >>>> >>>> 283 ? ? ? KSP ksp; >>>> 284 ? ? ? PC pc; >>>> 285 >>>> 286 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >>>> 287 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >>>> 288 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>> CHKERRQ(ierr); >>>> 289 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >>>> 290 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >>>> 291 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >>>> 292 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>> 293 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>> 294 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); >>>> 295 >>>> 296 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >>>> 297 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >>>> 298 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>> CHKERRQ(ierr); >>>> 299 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >>>> 300 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >>>> 301 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >>>> 302 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>> 303 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>> 304 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); >>> >>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>> making >>> struct YourShellContext { >>> ? PC fenpc,fwnpc; >>> ? Vec work; >>> ? PetscBool issetup; >>> }; >>> Then pass a pointer to this thing in when you create the PCShell. Then >>> something like >>> PCSetUp_YourShell(PC pc) { >>> struct YourShellContext *ctx; >>> Mat A,B; >>> MatStructure mstr; >>> const char *prefix; >>> char iprefix[256]; >>> PCShellGetContext(pc,&ctx); >>> PCGetOptionsPrefix(pc,&prefix); >>> if (!ctx->issetup) { >>> ? ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>> ? PCCreate(comm,&ctx->fenpc); >>> ? PCSetType(ctx->fenpc,PCILU); >>> ? PCFactorSetFill(ctx->fenpc,1); >>> ? PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>> ? snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>> ? PCSetOptionsPrefix(ctx->fenpc,iprefix); >>> ? PCSetFromOptions(ctx->fenpc); >>> ? /* Same as above for ctx->fwnpc */ >>> } >>> PCGetOperators(pc,&A,&B,&mstr); >>> PCSetOperators(ctx->fenpc,A,B,mstr); >>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>> PCSetUp(ctx->fenpc); >>> PCSetUp(ctx->fwnpc); >>> ctx->issetup = PETSC_TRUE; >>> } >>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>> ... >>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>> VecScale(Y,-1.0); >>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>> PCApply(ctx->fwnpc,ctx->work,Y); >>> } >>>> >>>> I don't know if this is what you intended... is there a way to make >>>> KSP recompute the factorization with the different ordering without >>>> destroying and recreating the KSP? >>> >>> Store both as in the code above. >>> >>>> >>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>> factorization function (and eventually the ones provided by external >>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>> need? If so, what's the best way to access it? >>> >>> Does the code above answer this question? >> >> >> >> -- >> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >> l'Anonymat" -- Non omnibus, sed mihi et tibi >> Amedeo Modigliani > > -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From bsmith at mcs.anl.gov Wed Jul 13 08:33:22 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 13 Jul 2011 08:33:22 -0500 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> Message-ID: <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: > Thanks Barry. > > What I'm doing at the moment in order to pass the ordering to the PC is > > PCFactorSetMatOrderingType(PC,"FESOrder"); > > where the FESOrder(...) function computes the ordering and has been > registered in the main function with > > MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); > > The same is done for al the different orderings I'm using inside the > PCSHELL suggested by Jed. > > My understanding is that the FESOrder(...) function is evaluated every > time the PCApply(...) function is called. Definitely not. Looking at PCSetUp_ILU() } else { if (!pc->setupcalled) { /* first time in so compute reordering and symbolic factorization */ ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. Just create a single KSP in the usual way and use the options -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse Barry > > If there was a way to store the IS containing the ordering ? or any > other array ? into the shell context (this I know how to do that) and > the pass the IS directly to the preconditioner instead of using > PCFactorSetMatOrderingType would be the best solution for me. > > Thanks > > Gianluca > > > On 12 July 2011 20:15, Barry Smith wrote: >> >> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: >> >>> Jed, thanks for your answer. >>> >>> This is exactly what I was looking for (it took me some time to test it...). >>> >>> Let me ask you a couple of additional questions: >>> >>> 1) in this configuration, is there an alternative way to pass the >>> ordering to the preconditioner? Becuse my ordering depends on the grid >>> an not on the solution, it is constant during all the computations and >>> I have no advantage on calling the function that creates multiple >>> times (the pc is built and destroyed several times). >> >> Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. >> >> PCShellSet/GetContext()? >>> >>> 2) what's the best way to make the shell pc selectable from the option >>> database, so that it can be replaced with more standard ones? >>> >> >> /* here the user can provide -pc_type shell or ilu or whatever */ >> KSPSetFromOptions(ksp); >> KSPGetPC(ksp,&pc); >> /* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ >> >> PCShellSetContext(pc,....) >> PCShellSetSetUp(pc,....) >> PCShellSetApply(pc,.....) >> >> >> Barry >> >> >>> Thanks >>> >>> Gianluca >>> >>> On 1 July 2011 14:43, Jed Brown wrote: >>>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>>> wrote: >>>>> >>>>> I've tried to implement Jed's solution as it was looking faster to >>>>> try. This is what I came up with for two different orders, FENOrder >>>>> and FWNOrder respectively >>>>> >>>>> 283 KSP ksp; >>>>> 284 PC pc; >>>>> 285 >>>>> 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>> 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>> 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>> CHKERRQ(ierr); >>>>> 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>> 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>> 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>> 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>>> 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>> 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>> 295 >>>>> 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>> 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>> 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>> CHKERRQ(ierr); >>>>> 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>> 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>> 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>> 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>>> 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>> 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>> >>>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>>> making >>>> struct YourShellContext { >>>> PC fenpc,fwnpc; >>>> Vec work; >>>> PetscBool issetup; >>>> }; >>>> Then pass a pointer to this thing in when you create the PCShell. Then >>>> something like >>>> PCSetUp_YourShell(PC pc) { >>>> struct YourShellContext *ctx; >>>> Mat A,B; >>>> MatStructure mstr; >>>> const char *prefix; >>>> char iprefix[256]; >>>> PCShellGetContext(pc,&ctx); >>>> PCGetOptionsPrefix(pc,&prefix); >>>> if (!ctx->issetup) { >>>> ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>>> PCCreate(comm,&ctx->fenpc); >>>> PCSetType(ctx->fenpc,PCILU); >>>> PCFactorSetFill(ctx->fenpc,1); >>>> PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>>> snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>>> PCSetOptionsPrefix(ctx->fenpc,iprefix); >>>> PCSetFromOptions(ctx->fenpc); >>>> /* Same as above for ctx->fwnpc */ >>>> } >>>> PCGetOperators(pc,&A,&B,&mstr); >>>> PCSetOperators(ctx->fenpc,A,B,mstr); >>>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>>> PCSetUp(ctx->fenpc); >>>> PCSetUp(ctx->fwnpc); >>>> ctx->issetup = PETSC_TRUE; >>>> } >>>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>>> ... >>>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>>> VecScale(Y,-1.0); >>>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>>> PCApply(ctx->fwnpc,ctx->work,Y); >>>> } >>>>> >>>>> I don't know if this is what you intended... is there a way to make >>>>> KSP recompute the factorization with the different ordering without >>>>> destroying and recreating the KSP? >>>> >>>> Store both as in the code above. >>>> >>>>> >>>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>>> factorization function (and eventually the ones provided by external >>>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>>> need? If so, what's the best way to access it? >>>> >>>> Does the code above answer this question? >>> >>> >>> >>> -- >>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>> Amedeo Modigliani >> >> > > > > -- > "[Je pense que] l'homme est un monde qui vaut des fois les mondes et > que les plus ardentes ambitions sont celles qui ont eu l'orgueil de > l'Anonymat" -- Non omnibus, sed mihi et tibi > Amedeo Modigliani From bsmith at mcs.anl.gov Wed Jul 13 08:55:17 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 13 Jul 2011 08:55:17 -0500 Subject: [petsc-users] Storing a matrix in PETSc binary form? In-Reply-To: References: Message-ID: On Jul 13, 2011, at 2:40 AM, John Chludzinski wrote: > $ ls -l > -rw-r--r-- 1 John None 192208072 Jul 13 03:31 binaryoutput > -rw-r--r-- 1 John None 0 Jul 13 03:31 binaryoutput.info > -rw-r--r-- 1 John None 128128032 Jul 13 01:43 DOF4002_k_double.bin > -rw-r--r-- 1 John None 128128032 Jul 13 01:43 DOF4002_m_double.bin > > where DOF4002_k_double.bin and DOF4002_m_double.bin are the file to be converted. > > What is binaryoutput.info (size=0)? It can store additional meta information about the matrix. You can ignore it. > And why is binaryoutput so much fatter than either DOF4002_k_double.bin or DOF4002_m_double.bin? The default PETSc binary storage is for sparse matrices so it stores both the nonzero values AND the column indices of the nonzero values (which are all of them). If you call PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE); before the MatView() then it will store the Petsc binary matrix without the column indices. BUT then you can only read it back in as a MATSEQDENSE or MATMPIDENSE matrix. Barry > > ---John > > On Wed, Jul 13, 2011 at 3:24 AM, John Chludzinski wrote: > Is this what you had in mind (more or less)? ---John > > //-----------------------------------CONVERTER--------------------------------------------------------------------------------------------------------- > > static char help[] = "Reads matrix: \n -f : file to load \n\n"; > > #define SIZE 4002 > > #include > #include > #include "petscmat.h" > > int main(int argc,char **args) > { > Mat A; > char file[PETSC_MAX_PATH_LEN]; > PetscErrorCode ierr; > PetscScalar *a; > PetscTruth flg; > PetscInt n=SIZE; > int i, j; > > PetscInitialize(&argc,&args,(char *)0,help); > > ierr = PetscOptionsGetString(PETSC_NULL,"-f",file,PETSC_MAX_PATH_LEN-1,&flg);CHKERRQ(ierr); > if (!flg) SETERRQ(1,"Must indicate binary file with the -f option"); > > ierr = PetscMalloc(SIZE*SIZE*sizeof(PetscScalar),&a);CHKERRQ(ierr); > > FILE *fpK; > > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > > if((fpK = fopen(file, "rb")) == NULL) > { > printf("Cannot open joe_DOF4002_k_double.bin\n"); > exit(1); > } > > fprintf(stderr, "Doubles read for K = %d\n", fread( (void *)K, sizeof(double), SIZE*SIZE, fpK )); > > for ( i = 0; i < SIZE; i++ ) > for ( j = i; j < SIZE; j++ ) a[i+j*SIZE]= *(K+i*SIZE+j); > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, a, &A); > > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > ierr = PetscFinalize();CHKERRQ(ierr); > return 0; > } > > > > On Wed, Jul 13, 2011 at 1:02 AM, Barry Smith wrote: > > If it is truly a dense matrix then it isn't clear that you want to store it as a PETSc binary matrix but you can using the following code > > PetscInt n; > PetscScalar *values; > > allocate values the correct size; read the entries into values then call MatCreateSeqDense(PETSC_COMM_SELF,n,n,values,&A); the call MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > Barry > > On Jul 12, 2011, at 11:41 PM, John Chludzinski wrote: > > > I have a matrix stored as a sequential list of binary values (not PETSc binary form) that I read in with a single block read: > > > > read( (void *)K, sizeof(double), SIZE*SIZE, fpK ). > > > > How do I convert this simple binary file of (sequentially stored) doubles into a "PETSc binary matrix file"? > > > > ---John > > > From clemens.domanig at uibk.ac.at Wed Jul 13 15:08:36 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Wed, 13 Jul 2011 22:08:36 +0200 Subject: [petsc-users] Debugging hints welcome Message-ID: <4E1DFB44.5000709@uibk.ac.at> Hi everyone, maybe some can offer som debugging-hints for my problem. My FEM-program uses a shell-element that has depending on the geometry 5 or 6 dof per node. The program uses MPI for parallel solving (LU, mumps). It works fine with all examples that have onyl 5 dof per node and that have a mixture of 5 and 6 dof per node. When doing examples that have 6 dof per node this happens: * when using more than 2 MPI processes everything seems to be fine. * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes This is the last output of -info, -mat_view_info, -vec_view_info (with 2 MPI processes, matrix size 1107648x1107648) [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. [0] MatStashScatterBegin_Private(): No of messages: 1 [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 704692232 [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: 24984360 unneeded,19875384 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. Using Inode routines Thx for your help - respectfully C. Domanig From knepley at gmail.com Wed Jul 13 15:10:57 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 13 Jul 2011 16:10:57 -0400 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1DFB44.5000709@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> Message-ID: On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig wrote: > Hi everyone, > > maybe some can offer som debugging-hints for my problem. > Its possible that there is a bug in the inode routines. Please try running with -mat_no_inode Thanks, Matt > My FEM-program uses a shell-element that has depending on the geometry 5 or > 6 dof per node. > > The program uses MPI for parallel solving (LU, mumps). > It works fine with all examples that have onyl 5 dof per node and that have > a mixture of 5 and 6 dof per node. > When doing examples that have 6 dof per node this happens: > * when using more than 2 MPI processes everything seems to be fine. > * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes > > This is the last output of -info, -mat_view_info, -vec_view_info (with 2 > MPI processes, matrix size 1107648x1107648) > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. > [0] MatStashScatterBegin_Private()**: No of messages: 1 > [0] MatStashScatterBegin_Private()**: Mesg_to: 1: size: 704692232 > [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: > 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. Using > Inode routines > > Thx for your help - respectfully C. Domanig > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Jul 13 15:12:11 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 13 Jul 2011 15:12:11 -0500 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1DFB44.5000709@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> Message-ID: On Wed, Jul 13, 2011 at 15:08, Clemens Domanig wrote: > The program uses MPI for parallel solving (LU, mumps). > It works fine with all examples that have onyl 5 dof per node and that have > a mixture of 5 and 6 dof per node. > When doing examples that have 6 dof per node this happens: > * when using more than 2 MPI processes everything seems to be fine. > * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes > It just hangs? Can you attach a debugger and get a stack trace? $ gdb -p PID [output from attaching] (gdb) bt full [send this output] -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemens.domanig at uibk.ac.at Wed Jul 13 15:56:24 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Wed, 13 Jul 2011 22:56:24 +0200 Subject: [petsc-users] Debugging hints welcome In-Reply-To: References: <4E1DFB44.5000709@uibk.ac.at> Message-ID: <4E1E0678.7020103@uibk.ac.at> I tried with -mat_no_inode - no effect. Thats the output [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. [0] MatStashScatterBegin_Private(): No of messages: 1 [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 704692232 [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: 24984360 unneeded,19875384 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 Am 2011-07-13 22:10, schrieb Matthew Knepley: > On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig > > wrote: > > Hi everyone, > > maybe some can offer som debugging-hints for my problem. > > > Its possible that there is a bug in the inode routines. Please try > running with -mat_no_inode > > Thanks, > > Matt > > My FEM-program uses a shell-element that has depending on the > geometry 5 or 6 dof per node. > > The program uses MPI for parallel solving (LU, mumps). > It works fine with all examples that have onyl 5 dof per node and > that have a mixture of 5 and 6 dof per node. > When doing examples that have 6 dof per node this happens: > * when using more than 2 MPI processes everything seems to be fine. > * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes > > This is the last output of -info, -mat_view_info, -vec_view_info > (with 2 MPI processes, matrix size 1107648x1107648) > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. > [0] MatStashScatterBegin_Private()__: No of messages: 1 > [0] MatStashScatterBegin_Private()__: Mesg_to: 1: size: 704692232 > [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 > mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage > space: 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. > Using Inode routines > > Thx for your help - respectfully C. Domanig > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener From bsmith at mcs.anl.gov Wed Jul 13 15:59:30 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 13 Jul 2011 15:59:30 -0500 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1E0678.7020103@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> Message-ID: If it really hangs on one process then just run with the option -start_in_debugger noxterm and type cont in the debugger; when you think it is hanging (after a few minutes I guess) hit control-c and then type where to see where it is "hanging". My guess is that it is not hanging but with the size dof your preallocation is way off and it is just taking a huge amount of time to set the values. Suggest revisit where you determine the preallocation. Barry On Jul 13, 2011, at 3:56 PM, Clemens Domanig wrote: > I tried with -mat_no_inode - no effect. Thats the output > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. > [0] MatStashScatterBegin_Private(): No of messages: 1 > [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 704692232 > [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > > > Am 2011-07-13 22:10, schrieb Matthew Knepley: >> On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig >> > wrote: >> >> Hi everyone, >> >> maybe some can offer som debugging-hints for my problem. >> >> >> Its possible that there is a bug in the inode routines. Please try >> running with -mat_no_inode >> >> Thanks, >> >> Matt >> >> My FEM-program uses a shell-element that has depending on the >> geometry 5 or 6 dof per node. >> >> The program uses MPI for parallel solving (LU, mumps). >> It works fine with all examples that have onyl 5 dof per node and >> that have a mixture of 5 and 6 dof per node. >> When doing examples that have 6 dof per node this happens: >> * when using more than 2 MPI processes everything seems to be fine. >> * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes >> >> This is the last output of -info, -mat_view_info, -vec_view_info >> (with 2 MPI processes, matrix size 1107648x1107648) >> >> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >> [0] MatStashScatterBegin_Private()__: No of messages: 1 >> [0] MatStashScatterBegin_Private()__: Mesg_to: 1: size: 704692232 >> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 >> mallocs. >> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage >> space: 24984360 unneeded,19875384 used >> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() >> is 0 >> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >> [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. >> Using Inode routines >> >> Thx for your help - respectfully C. Domanig >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener > From clemens.domanig at uibk.ac.at Wed Jul 13 16:08:17 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Wed, 13 Jul 2011 23:08:17 +0200 Subject: [petsc-users] Debugging hints welcome In-Reply-To: References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> Message-ID: <4E1E0941.5080405@uibk.ac.at> I preallocate a sparse matrix 1107648x1107648 at the beginning. I checked all indices when filling the matrix with MatSetValues and they are all smaller than 1107648. Just to give you some time measurements: 1 MPI - "hangs" 2 MPI - "hangs" even after 20 min no change 3 MPI - 21s for assembly 4 MPI - 19s for assembly will test with -start_in_debugger tomorrow morning thx Am 2011-07-13 22:59, schrieb Barry Smith: > > If it really hangs on one process then just run with the option -start_in_debugger noxterm and type cont in the debugger; when you think it is hanging (after a few minutes I guess) hit control-c and then type where to see where it is "hanging". My guess is that it is not hanging but with the size dof your preallocation is way off and it is just taking a huge amount of time to set the values. Suggest revisit where you determine the preallocation. > > > Barry > > On Jul 13, 2011, at 3:56 PM, Clemens Domanig wrote: > >> I tried with -mat_no_inode - no effect. Thats the output >> >> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >> [0] MatStashScatterBegin_Private(): No of messages: 1 >> [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 704692232 >> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. >> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: 24984360 unneeded,19875384 used >> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 >> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >> >> >> Am 2011-07-13 22:10, schrieb Matthew Knepley: >>> On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig >>> > wrote: >>> >>> Hi everyone, >>> >>> maybe some can offer som debugging-hints for my problem. >>> >>> >>> Its possible that there is a bug in the inode routines. Please try >>> running with -mat_no_inode >>> >>> Thanks, >>> >>> Matt >>> >>> My FEM-program uses a shell-element that has depending on the >>> geometry 5 or 6 dof per node. >>> >>> The program uses MPI for parallel solving (LU, mumps). >>> It works fine with all examples that have onyl 5 dof per node and >>> that have a mixture of 5 and 6 dof per node. >>> When doing examples that have 6 dof per node this happens: >>> * when using more than 2 MPI processes everything seems to be fine. >>> * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes >>> >>> This is the last output of -info, -mat_view_info, -vec_view_info >>> (with 2 MPI processes, matrix size 1107648x1107648) >>> >>> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >>> [0] MatStashScatterBegin_Private()__: No of messages: 1 >>> [0] MatStashScatterBegin_Private()__: Mesg_to: 1: size: 704692232 >>> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 >>> mallocs. >>> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage >>> space: 24984360 unneeded,19875384 used >>> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() >>> is 0 >>> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >>> [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. >>> Using Inode routines >>> >>> Thx for your help - respectfully C. Domanig >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which >>> their experiments lead. >>> -- Norbert Wiener >> > From bsmith at mcs.anl.gov Wed Jul 13 16:25:36 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 13 Jul 2011 16:25:36 -0500 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1E0941.5080405@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> <4E1E0941.5080405@uibk.ac.at> Message-ID: Take a look at http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#efficient-assembly On Jul 13, 2011, at 4:08 PM, Clemens Domanig wrote: > I preallocate a sparse matrix 1107648x1107648 at the beginning. I checked all indices when filling the matrix with MatSetValues and they are all smaller than 1107648. > Just to give you some time measurements: > 1 MPI - "hangs" > 2 MPI - "hangs" even after 20 min no change > 3 MPI - 21s for assembly > 4 MPI - 19s for assembly > > will test with -start_in_debugger tomorrow morning > thx > > Am 2011-07-13 22:59, schrieb Barry Smith: >> >> If it really hangs on one process then just run with the option -start_in_debugger noxterm and type cont in the debugger; when you think it is hanging (after a few minutes I guess) hit control-c and then type where to see where it is "hanging". My guess is that it is not hanging but with the size dof your preallocation is way off and it is just taking a huge amount of time to set the values. Suggest revisit where you determine the preallocation. >> >> >> Barry >> >> On Jul 13, 2011, at 3:56 PM, Clemens Domanig wrote: >> >>> I tried with -mat_no_inode - no effect. Thats the output >>> >>> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >>> [0] MatStashScatterBegin_Private(): No of messages: 1 >>> [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 704692232 >>> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. >>> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: 24984360 unneeded,19875384 used >>> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 >>> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >>> >>> >>> Am 2011-07-13 22:10, schrieb Matthew Knepley: >>>> On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig >>>> > wrote: >>>> >>>> Hi everyone, >>>> >>>> maybe some can offer som debugging-hints for my problem. >>>> >>>> >>>> Its possible that there is a bug in the inode routines. Please try >>>> running with -mat_no_inode >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> My FEM-program uses a shell-element that has depending on the >>>> geometry 5 or 6 dof per node. >>>> >>>> The program uses MPI for parallel solving (LU, mumps). >>>> It works fine with all examples that have onyl 5 dof per node and >>>> that have a mixture of 5 and 6 dof per node. >>>> When doing examples that have 6 dof per node this happens: >>>> * when using more than 2 MPI processes everything seems to be fine. >>>> * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes >>>> >>>> This is the last output of -info, -mat_view_info, -vec_view_info >>>> (with 2 MPI processes, matrix size 1107648x1107648) >>>> >>>> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >>>> [0] MatStashScatterBegin_Private()__: No of messages: 1 >>>> [0] MatStashScatterBegin_Private()__: Mesg_to: 1: size: 704692232 >>>> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 >>>> mallocs. >>>> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage >>>> space: 24984360 unneeded,19875384 used >>>> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() >>>> is 0 >>>> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >>>> [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. >>>> Using Inode routines >>>> >>>> Thx for your help - respectfully C. Domanig >>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which >>>> their experiments lead. >>>> -- Norbert Wiener >>> >> > From knepley at gmail.com Wed Jul 13 21:53:02 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 13 Jul 2011 22:53:02 -0400 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1E0678.7020103@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> Message-ID: On Wed, Jul 13, 2011 at 4:56 PM, Clemens Domanig wrote: > I tried with -mat_no_inode - no effect. Thats the output > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. > [0] MatStashScatterBegin_Private()**: No of messages: 1 > [0] MatStashScatterBegin_Private()**: Mesg_to: 1: size: 704692232 > ^^^^^^^^^^ Do you really mean to set 700M of off-process values? I think Barry is correct that it is just taking forever to send this. Matt [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage space: > 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > > > Am 2011-07-13 22:10, schrieb Matthew Knepley: > >> On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig >> >> >> wrote: >> >> Hi everyone, >> >> maybe some can offer som debugging-hints for my problem. >> >> >> Its possible that there is a bug in the inode routines. Please try >> running with -mat_no_inode >> >> Thanks, >> >> Matt >> >> My FEM-program uses a shell-element that has depending on the >> geometry 5 or 6 dof per node. >> >> The program uses MPI for parallel solving (LU, mumps). >> It works fine with all examples that have onyl 5 dof per node and >> that have a mixture of 5 and 6 dof per node. >> When doing examples that have 6 dof per node this happens: >> * when using more than 2 MPI processes everything seems to be fine. >> * when using 1 or 2 MPI processes MatAssemblyBegin() never finishes >> >> This is the last output of -info, -mat_view_info, -vec_view_info >> (with 2 MPI processes, matrix size 1107648x1107648) >> >> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >> [0] MatStashScatterBegin_Private()**__: No of messages: 1 >> [0] MatStashScatterBegin_Private()**__: Mesg_to: 1: size: 704692232 >> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 >> mallocs. >> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage >> space: 24984360 unneeded,19875384 used >> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() >> is 0 >> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >> [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit used: 5. >> Using Inode routines >> >> Thx for your help - respectfully C. Domanig >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemens.domanig at uibk.ac.at Thu Jul 14 00:44:07 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Thu, 14 Jul 2011 07:44:07 +0200 Subject: [petsc-users] Debugging hints welcome In-Reply-To: References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> Message-ID: <4E1E8227.3060002@uibk.ac.at> I know it is lots of values I send but this will only run on a shared memory system. And to me it is strange that it is just ~20s with 3+ MPI-proc. Matthew Knepley wrote: > On Wed, Jul 13, 2011 at 4:56 PM, Clemens Domanig > > wrote: > > I tried with -mat_no_inode - no effect. Thats the output > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. > [0] MatStashScatterBegin_Private()__: No of messages: 1 > [0] MatStashScatterBegin_Private()__: Mesg_to: 1: size: 704692232 > > > ^^^^^^^^^^ Do you really mean to set 700M of > off-process values? > > I think Barry is correct that it is just taking forever to send this. > > Matt > > [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 > mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage > space: 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() > is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > > > Am 2011-07-13 22:10, schrieb Matthew Knepley: > > On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig > > >> wrote: > > Hi everyone, > > maybe some can offer som debugging-hints for my problem. > > > Its possible that there is a bug in the inode routines. Please try > running with -mat_no_inode > > Thanks, > > Matt > > My FEM-program uses a shell-element that has depending on the > geometry 5 or 6 dof per node. > > The program uses MPI for parallel solving (LU, mumps). > It works fine with all examples that have onyl 5 dof per node and > that have a mixture of 5 and 6 dof per node. > When doing examples that have 6 dof per node this happens: > * when using more than 2 MPI processes everything seems to be > fine. > * when using 1 or 2 MPI processes MatAssemblyBegin() never > finishes > > This is the last output of -info, -mat_view_info, -vec_view_info > (with 2 MPI processes, matrix size 1107648x1107648) > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 > mallocs. > [0] MatStashScatterBegin_Private()____: No of messages: 1 > [0] MatStashScatterBegin_Private()____: Mesg_to: 1: size: > 704692232 > [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, > uses 13 > mallocs. > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; > storage > space: 24984360 unneeded,19875384 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during > MatSetValues() > is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 > [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit > used: 5. > Using Inode routines > > Thx for your help - respectfully C. Domanig > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener From knepley at gmail.com Thu Jul 14 06:21:50 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 14 Jul 2011 07:21:50 -0400 Subject: [petsc-users] Debugging hints welcome In-Reply-To: <4E1E8227.3060002@uibk.ac.at> References: <4E1DFB44.5000709@uibk.ac.at> <4E1E0678.7020103@uibk.ac.at> <4E1E8227.3060002@uibk.ac.at> Message-ID: On Thu, Jul 14, 2011 at 1:44 AM, Clemens Domanig wrote: > I know it is lots of values I send but this will only run on a shared > memory system. And to me it is strange that it is just ~20s with 3+ > MPI-proc. > You could have hit a level in the memory hierarchy, where it starts swapping to disk. To definitely say would take a lot of work. However, its very easy to definitely say whether there is deadlock. As both Barry and Jed said, please connect with the debugger and look at the stack trace. Matt > Matthew Knepley wrote: > > On Wed, Jul 13, 2011 at 4:56 PM, Clemens Domanig < >> clemens.domanig at uibk.ac.at >> >> wrote: >> >> I tried with -mat_no_inode - no effect. Thats the output >> >> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs. >> [0] MatStashScatterBegin_Private()**__: No of messages: 1 >> [0] MatStashScatterBegin_Private()**__: Mesg_to: 1: size: 704692232 >> >> >> ^^^^^^^^^^ Do you really mean to set 700M of off-process >> values? >> >> I think Barry is correct that it is just taking forever to send this. >> >> Matt >> >> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13 >> mallocs. >> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage >> space: 24984360 unneeded,19875384 used >> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() >> is 0 >> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >> >> >> Am 2011-07-13 22:10, schrieb Matthew Knepley: >> >> On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig >> >> > >> > >>> >> wrote: >> >> Hi everyone, >> >> maybe some can offer som debugging-hints for my problem. >> >> >> Its possible that there is a bug in the inode routines. Please try >> running with -mat_no_inode >> >> Thanks, >> >> Matt >> >> My FEM-program uses a shell-element that has depending on the >> geometry 5 or 6 dof per node. >> >> The program uses MPI for parallel solving (LU, mumps). >> It works fine with all examples that have onyl 5 dof per node >> and >> that have a mixture of 5 and 6 dof per node. >> When doing examples that have 6 dof per node this happens: >> * when using more than 2 MPI processes everything seems to be >> fine. >> * when using 1 or 2 MPI processes MatAssemblyBegin() never >> finishes >> >> This is the last output of -info, -mat_view_info, -vec_view_info >> (with 2 MPI processes, matrix size 1107648x1107648) >> >> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 >> mallocs. >> [0] MatStashScatterBegin_Private()**____: No of messages: 1 >> [0] MatStashScatterBegin_Private()**____: Mesg_to: 1: size: >> 704692232 >> [0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, >> uses 13 >> mallocs. >> [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; >> storage >> space: 24984360 unneeded,19875384 used >> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during >> MatSetValues() >> is 0 >> [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42 >> [0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit >> used: 5. >> Using Inode routines >> >> Thx for your help - respectfully C. Domanig >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which >> their experiments lead. >> -- Norbert Wiener >> >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Kevin.Green at uoit.ca Thu Jul 14 12:31:34 2011 From: Kevin.Green at uoit.ca (Kevin Green) Date: Thu, 14 Jul 2011 13:31:34 -0400 Subject: [petsc-users] Field dependent stencil in DAs In-Reply-To: References: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local>, Message-ID: <8B5CE3730B35A9449FE54172FFD223F002D940C99669@ITOMSCPP01.oncampus.local> ________________________________________ From: petsc-users-bounces at mcs.anl.gov [petsc-users-bounces at mcs.anl.gov] On Behalf Of Jed Brown [jedbrown at mcs.anl.gov] Sent: Monday, July 11, 2011 7:38 PM To: PETSc users list Subject: Re: [petsc-users] Field dependent stencil in DAs On Mon, Jul 11, 2011 at 18:28, Kevin Green > wrote: Essentially, I'm (currently) looking to numerically solve a system of 5 time dependent equations, but only 1 of the fields is involved with spatial derivatives. Now, I've found a couple of ways to do this, either by introducing a Field struct cf. ${PETSC_DIR}/src/snes/examples/tutorials/ex19.c, or by changing one of the various ts examples to use DAVecGetArrayDOF(...). Would there be any performance difference between introducing a Field struct compared to DAVecGetArrayDOF(...)? Intuitively, I would think not...but I suppose I should ask since I'm here anyway. I like using field structs because then I get to use names instead of numbers for my fields and because there is one less level of pointer indirection. Now the real reason I came: Is there some sort of mask that can be applied to the DAs that would turn off the communication of ghost points for some degrees of freedom? I assume that the communication structures are set up so that only a single message needs to be passed between communicating processes on update, but this still results in passing 5X the data that I have to. Any comment on this would be much appreciated. DMDASetBlockFills() Thank you Jed, this is exactly what I was looking for. In the most recent version, it appears to be named as just DASetBlockFills(). From knepley at gmail.com Thu Jul 14 12:42:08 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 14 Jul 2011 17:42:08 +0000 Subject: [petsc-users] Field dependent stencil in DAs In-Reply-To: <8B5CE3730B35A9449FE54172FFD223F002D940C99669@ITOMSCPP01.oncampus.local> References: <8B5CE3730B35A9449FE54172FFD223F002D940C99665@ITOMSCPP01.oncampus.local> <8B5CE3730B35A9449FE54172FFD223F002D940C99669@ITOMSCPP01.oncampus.local> Message-ID: On Thu, Jul 14, 2011 at 5:31 PM, Kevin Green wrote: > > ________________________________________ > From: petsc-users-bounces at mcs.anl.gov [petsc-users-bounces at mcs.anl.gov] On > Behalf Of Jed Brown [jedbrown at mcs.anl.gov] > Sent: Monday, July 11, 2011 7:38 PM > To: PETSc users list > Subject: Re: [petsc-users] Field dependent stencil in DAs > > On Mon, Jul 11, 2011 at 18:28, Kevin Green Kevin.Green at uoit.ca>> wrote: > Essentially, I'm (currently) looking to numerically solve a system of 5 > time dependent equations, but only 1 of the fields is involved with spatial > derivatives. Now, I've found a couple of ways to do this, either by > introducing a Field struct cf. > ${PETSC_DIR}/src/snes/examples/tutorials/ex19.c, or by changing one of the > various ts examples to use DAVecGetArrayDOF(...). Would there be any > performance difference between introducing a Field struct compared to > DAVecGetArrayDOF(...)? Intuitively, I would think not...but I suppose I > should ask since I'm here anyway. > > I like using field structs because then I get to use names instead of > numbers for my fields and because there is one less level of pointer > indirection. > > > Now the real reason I came: Is there some sort of mask that can be applied > to the DAs that would turn off the communication of ghost points for some > degrees of freedom? I assume that the communication structures are set up > so that only a single message needs to be passed between communicating > processes on update, but this still results in passing 5X the data that I > have to. Any comment on this would be much appreciated. > > DMDASetBlockFills() > > > Thank you Jed, this is exactly what I was looking for. In the most recent > version, it appears to be named as just DASetBlockFills(). > Its DMDASetBlockFills() in petsc-dev, and DASetBlockFills() in petsc 3.1 Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Thu Jul 14 13:18:36 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 14 Jul 2011 11:18:36 -0700 Subject: [petsc-users] To store the matrix. Message-ID: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Dear all, I have assembled a 2d matrix through the FormJacobianLocal() routine. I would like to store this sparse MXN matrix A in the following form: .................................................... line1: M N numberofnonzeros line2: i j value line3: i j value ... ... ... .................................................... The first line gives the size of the matrix and the number of nonzeros and starting from the second line, the index of the nonzero and its value is shown, too. Thanks very much! Best, Rebecca From jedbrown at mcs.anl.gov Thu Jul 14 13:28:32 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 14 Jul 2011 13:28:32 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan wrote: > I would like to store this sparse MXN matrix A in the following form: > .................................................... > > line1: M N numberofnonzeros > line2: i j value > line3: i j value > This "COO" format is not an efficient representation. Why do you want to store it this way? (You can produce this by getting the AIJ structure, allocating a new array for the row indices, and filling it redundantly from the AIJ structure. -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Thu Jul 14 13:30:41 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 14 Jul 2011 11:30:41 -0700 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> Hi Jed, I need to have this form for the simple input for PDSLin. What do you mean by "COO"? Any examples to show how to pull that new array for reading from the AIJ structure? Thanks a lot! R On Jul 14, 2011, at 11:28 AM, Jed Brown wrote: > On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan wrote: > I would like to store this sparse MXN matrix A in the following form: > .................................................... > > line1: M N numberofnonzeros > line2: i j value > line3: i j value > > This "COO" format is not an efficient representation. Why do you want to store it this way? > > (You can produce this by getting the AIJ structure, allocating a new array for the row indices, and filling it redundantly from the AIJ structure. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Jul 14 13:36:09 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 14 Jul 2011 13:36:09 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> Message-ID: On Thu, Jul 14, 2011 at 13:30, Xuefei (Rebecca) Yuan wrote: > I need to have this form for the simple input for PDSLin. > > What do you mean by "COO"? > http://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_.28COO.29 > Any examples to show how to pull that new array for reading from the AIJ > structure? > Get the indices using MatGetRowIJ() and the values using MatGetArray(). -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Jul 14 13:36:23 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 14 Jul 2011 18:36:23 +0000 Subject: [petsc-users] To store the matrix. In-Reply-To: <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> Message-ID: On Thu, Jul 14, 2011 at 6:30 PM, Xuefei (Rebecca) Yuan wrote: > Hi Jed, > > I need to have this form for the simple input for PDSLin. > > What do you mean by "COO"? > > Any examples to show how to pull that new array for reading from the AIJ > structure? > First use MatGetSubmatrix() to pull the whole matrix into proc 0 if you are in parallel. Then you MatGetRow() for each row, and write each entry in your form. Matt > Thanks a lot! > > R > > On Jul 14, 2011, at 11:28 AM, Jed Brown wrote: > > On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan wrote: > >> I would like to store this sparse MXN matrix A in the following form: >> .................................................... >> >> line1: M N numberofnonzeros >> line2: i j value >> line3: i j value >> > > This "COO" format is not an efficient representation. Why do you want to > store it this way? > > (You can produce this by getting the AIJ structure, allocating a new array > for the row indices, and filling it redundantly from the AIJ structure. > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Thu Jul 14 13:37:55 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 14 Jul 2011 11:37:55 -0700 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> Message-ID: <9FB81950-4151-4154-8659-C9030CBB44D2@lbl.gov> Dear Matt and Jed, Thanks very much! That helps! Best regards, Rebecca On Jul 14, 2011, at 11:36 AM, Matthew Knepley wrote: > On Thu, Jul 14, 2011 at 6:30 PM, Xuefei (Rebecca) Yuan wrote: > Hi Jed, > > I need to have this form for the simple input for PDSLin. > > What do you mean by "COO"? > > Any examples to show how to pull that new array for reading from the AIJ structure? > > First use MatGetSubmatrix() to pull the whole matrix into proc 0 if you are in parallel. > Then you MatGetRow() for each row, and write each entry in your form. > > Matt > > Thanks a lot! > > R > > On Jul 14, 2011, at 11:28 AM, Jed Brown wrote: > >> On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan wrote: >> I would like to store this sparse MXN matrix A in the following form: >> .................................................... >> >> line1: M N numberofnonzeros >> line2: i j value >> line3: i j value >> >> This "COO" format is not an efficient representation. Why do you want to store it this way? >> >> (You can produce this by getting the AIJ structure, allocating a new array for the row indices, and filling it redundantly from the AIJ structure. > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From vijay.m at gmail.com Thu Jul 14 13:39:46 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Thu, 14 Jul 2011 13:39:46 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: Jed, I think the format is useful to serialize a matrix into MATLAB format. Typically, you could do something like nnz = 1 ; index = zeros(nnz,3); index = [ 1 1 1.0 ]; A = spconvert(index); I am unsure if PETSC_VIEWER_MATLAB already provides this functionality for matrices. The document for the viewer specifies that currently only vectors are stored in .mat files and PetscBinaryRead needs to be used for matrices. This will definitely be faster to load for larger matrices. Vijay On Thu, Jul 14, 2011 at 1:28 PM, Jed Brown wrote: > On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan wrote: >> >> I would like to store this sparse MXN matrix A in the following form: >> .................................................... >> >> line1: ?M ? ? ? N ? ? ? numberofnonzeros >> line2: ?i ? ? ? j ? ? ? value >> line3: ?i ? ? ? j ? ? ? value > > This "COO" format is not an efficient representation. Why do you want to > store it this way? > (You can produce this by getting the AIJ structure, allocating a new array > for the row indices, and filling it redundantly from the AIJ structure. From jedbrown at mcs.anl.gov Thu Jul 14 13:45:17 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 14 Jul 2011 13:45:17 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: On Thu, Jul 14, 2011 at 13:39, Vijay S. Mahadevan wrote: > I think the format is useful to serialize a matrix into MATLAB format. > Yes, but a binary viewer should be used for this instead. > I am unsure if PETSC_VIEWER_MATLAB already provides this functionality > for matrices. > You mean the ASCII MATLAB format? You can view binary matrices to Matlab without any text representation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vijay.m at gmail.com Thu Jul 14 13:52:30 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Thu, 14 Jul 2011 13:52:30 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: > Yes, but a binary viewer should be used for this instead. Agreed. But for small problems in debug scenarios, it is useful to check the entries in ascii format. > You mean the ASCII MATLAB format? You can view binary matrices to Matlab > without any text representation. Yes. Do you mean the PetscBinaryRead/Write approach for storing in matlab readable binary ? On Thu, Jul 14, 2011 at 1:45 PM, Jed Brown wrote: > On Thu, Jul 14, 2011 at 13:39, Vijay S. Mahadevan wrote: >> >> I think the format is useful to serialize a matrix into MATLAB format. > > Yes, but a binary viewer should be used for this instead. > >> >> I am unsure if PETSC_VIEWER_MATLAB already provides this functionality >> for matrices. > > You mean the ASCII MATLAB format? You can view binary matrices to Matlab > without any text representation. From jedbrown at mcs.anl.gov Thu Jul 14 14:00:08 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 14 Jul 2011 14:00:08 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> Message-ID: On Thu, Jul 14, 2011 at 13:52, Vijay S. Mahadevan wrote: > Yes. Do you mean the PetscBinaryRead/Write approach for storing in > matlab readable binary ? > Yes, or using PetscMatlabEnginePut() to talk directly to a running Matlab session. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Thu Jul 14 14:05:42 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 14 Jul 2011 14:05:42 -0500 Subject: [petsc-users] To store the matrix. In-Reply-To: <9FB81950-4151-4154-8659-C9030CBB44D2@lbl.gov> References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> <9FB81950-4151-4154-8659-C9030CBB44D2@lbl.gov> Message-ID: Rebecca, We have private routines MatConvertToTriples_xxx_xxx() in ~petsc/src/mat/impls/aij/mpi/mumps/mumps.c which convert petsc seqaij and mpiaij to triples: row[], col[], val[] . You may take a look at these routines. Hong On Thu, Jul 14, 2011 at 1:37 PM, Xuefei (Rebecca) Yuan wrote: > Dear Matt and Jed, > Thanks very much! That helps! > Best regards, > Rebecca > > > On Jul 14, 2011, at 11:36 AM, Matthew Knepley wrote: > > On Thu, Jul 14, 2011 at 6:30 PM, Xuefei (Rebecca) Yuan > wrote: >> >> Hi Jed, >> I need to have this form for the simple input for PDSLin. >> What do you mean by "COO"? >> Any examples to show how to pull that new array for reading from the AIJ >> structure? > > First use MatGetSubmatrix() to pull the whole matrix into proc 0 if you are > in parallel. > Then you MatGetRow() for each row, and write each entry in your form. > ? ?Matt > >> >> Thanks a lot! >> R >> On Jul 14, 2011, at 11:28 AM, Jed Brown wrote: >> >> On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan >> wrote: >>> >>> I would like to store this sparse MXN matrix A in the following form: >>> .................................................... >>> >>> line1: ?M ? ? ? N ? ? ? numberofnonzeros >>> line2: ?i ? ? ? j ? ? ? value >>> line3: ?i ? ? ? j ? ? ? value >> >> This "COO" format is not an efficient representation. Why do you want to >> store it this way? >> (You can produce this by getting the AIJ structure, allocating a new array >> for the row indices, and filling it redundantly from the AIJ structure. > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > > From xyuan at lbl.gov Thu Jul 14 14:19:12 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 14 Jul 2011 12:19:12 -0700 Subject: [petsc-users] To store the matrix. In-Reply-To: References: <008B2356-A64B-4BA7-B7E5-1866AE99C740@lbl.gov> <2BAB4456-ED18-4AF0-87B1-B0337649C930@lbl.gov> <9FB81950-4151-4154-8659-C9030CBB44D2@lbl.gov> Message-ID: Dear Hong, Thanks very much! I have aij, so it is the MatConvertToTriples_seqaij_seqaij() I need to call after get the global matrix? Thanks, Rebecca On Jul 14, 2011, at 12:05 PM, Hong Zhang wrote: > Rebecca, > > We have private routines MatConvertToTriples_xxx_xxx() > in ~petsc/src/mat/impls/aij/mpi/mumps/mumps.c > which convert petsc seqaij and mpiaij to triples: row[], col[], val[] . > You may take a look at these routines. > > Hong > > On Thu, Jul 14, 2011 at 1:37 PM, Xuefei (Rebecca) Yuan wrote: >> Dear Matt and Jed, >> Thanks very much! That helps! >> Best regards, >> Rebecca >> >> >> On Jul 14, 2011, at 11:36 AM, Matthew Knepley wrote: >> >> On Thu, Jul 14, 2011 at 6:30 PM, Xuefei (Rebecca) Yuan >> wrote: >>> >>> Hi Jed, >>> I need to have this form for the simple input for PDSLin. >>> What do you mean by "COO"? >>> Any examples to show how to pull that new array for reading from the AIJ >>> structure? >> >> First use MatGetSubmatrix() to pull the whole matrix into proc 0 if you are >> in parallel. >> Then you MatGetRow() for each row, and write each entry in your form. >> Matt >> >>> >>> Thanks a lot! >>> R >>> On Jul 14, 2011, at 11:28 AM, Jed Brown wrote: >>> >>> On Thu, Jul 14, 2011 at 13:18, Xuefei (Rebecca) Yuan >>> wrote: >>>> >>>> I would like to store this sparse MXN matrix A in the following form: >>>> .................................................... >>>> >>>> line1: M N numberofnonzeros >>>> line2: i j value >>>> line3: i j value >>> >>> This "COO" format is not an efficient representation. Why do you want to >>> store it this way? >>> (You can produce this by getting the AIJ structure, allocating a new array >>> for the row indices, and filling it redundantly from the AIJ structure. >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> >> From ecoon at lanl.gov Thu Jul 14 14:22:13 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Thu, 14 Jul 2011 13:22:13 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> Message-ID: <1310671333.22449.180.camel@echo.lanl.gov> Attached patch takes Lisandro's matio.py and puts it in a PetscBinaryRead.py that reads a binary with (potentially) multiple Petsc objects, deciphers the contents of the file from the header, and returns numpy objects. It's basically the same as the PetscBinaryRead.m I've tested this on Vecs, but not Mats or ISs, and I haven't thoroughly tested degenerate cases. Will do some testing, but I wanted Lisandro to take a look at it as well... Thanks, Ethan On Wed, 2011-07-06 at 12:24 -0300, Lisandro Dalcin wrote: > On 6 July 2011 11:41, Ethan Coon wrote: > > Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. > > > > Using petsc4py for reading PETSc binary formats is overkill. All this > can be done with just NumPy. Should PETSc ship some Python code to > read IS's, Vec's, Mat's in binary format? Can any of you suggest an > appropriate location in the source tree? > > > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: PetscBinaryRead.py Type: text/x-python Size: 4791 bytes Desc: not available URL: From bsmith at mcs.anl.gov Thu Jul 14 14:38:23 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 14 Jul 2011 14:38:23 -0500 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1310671333.22449.180.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <1310671333.22449.180.camel@echo.lanl.gov> Message-ID: Ethan, Excellent. I have the matlab programs in a subdirectory called matlab. Will there be several of these? So should we have a bin/python subdirectory for them? Barry On Jul 14, 2011, at 2:22 PM, Ethan Coon wrote: > Attached patch takes Lisandro's matio.py and puts it in a > PetscBinaryRead.py that reads a binary with (potentially) multiple Petsc > objects, deciphers the contents of the file from the header, and returns > numpy objects. It's basically the same as the PetscBinaryRead.m > > I've tested this on Vecs, but not Mats or ISs, and I haven't thoroughly > tested degenerate cases. Will do some testing, but I wanted Lisandro to > take a look at it as well... > > Thanks, > > Ethan > > On Wed, 2011-07-06 at 12:24 -0300, Lisandro Dalcin wrote: >> On 6 July 2011 11:41, Ethan Coon wrote: >>> Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. >>> >> >> Using petsc4py for reading PETSc binary formats is overkill. All this >> can be done with just NumPy. Should PETSc ship some Python code to >> read IS's, Vec's, Mat's in binary format? Can any of you suggest an >> appropriate location in the source tree? >> >> >> >> > > -- > ------------------------------------ > Ethan Coon > Post-Doctoral Researcher > Applied Mathematics - T-5 > Los Alamos National Laboratory > 505-665-8289 > > http://www.ldeo.columbia.edu/~ecoon/ > ------------------------------------ > From ecoon at lanl.gov Thu Jul 14 14:45:56 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Thu, 14 Jul 2011 13:45:56 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <1310671333.22449.180.camel@echo.lanl.gov> Message-ID: <1310672756.22449.183.camel@echo.lanl.gov> I suppose it's possible, eventually. Much of the matlab stuff is related to the socket and the "wrappers", which exist in petsc4py instead. There may be a few though -- PetscBinaryWrite.py at least. Ethan On Thu, 2011-07-14 at 14:38 -0500, Barry Smith wrote: > Ethan, > > Excellent. > > I have the matlab programs in a subdirectory called matlab. Will there be several of these? So should we have a bin/python subdirectory for them? > > Barry > > > On Jul 14, 2011, at 2:22 PM, Ethan Coon wrote: > > > Attached patch takes Lisandro's matio.py and puts it in a > > PetscBinaryRead.py that reads a binary with (potentially) multiple Petsc > > objects, deciphers the contents of the file from the header, and returns > > numpy objects. It's basically the same as the PetscBinaryRead.m > > > > I've tested this on Vecs, but not Mats or ISs, and I haven't thoroughly > > tested degenerate cases. Will do some testing, but I wanted Lisandro to > > take a look at it as well... > > > > Thanks, > > > > Ethan > > > > On Wed, 2011-07-06 at 12:24 -0300, Lisandro Dalcin wrote: > >> On 6 July 2011 11:41, Ethan Coon wrote: > >>> Both Paraview and Visit generally make ugly axes/colorbars/keys/etc. In my opinion they both look fine for presentations and my own viewing, but are not really acceptable for publication-quality. For things run on a reasonably-sized problem, matplotlib (using either h5py to read hdf5, pyvtk to read vtk, or petsc4py to read PETSc's binary format) makes publishable plots. > >>> > >> > >> Using petsc4py for reading PETSc binary formats is overkill. All this > >> can be done with just NumPy. Should PETSc ship some Python code to > >> read IS's, Vec's, Mat's in binary format? Can any of you suggest an > >> appropriate location in the source tree? > >> > >> > >> > >> > > > > -- > > ------------------------------------ > > Ethan Coon > > Post-Doctoral Researcher > > Applied Mathematics - T-5 > > Los Alamos National Laboratory > > 505-665-8289 > > > > http://www.ldeo.columbia.edu/~ecoon/ > > ------------------------------------ > > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From dalcinl at gmail.com Thu Jul 14 15:47:12 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 14 Jul 2011 17:47:12 -0300 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1310671333.22449.180.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <1310671333.22449.180.camel@echo.lanl.gov> Message-ID: On 14 July 2011 16:22, Ethan Coon wrote: > Attached patch takes Lisandro's matio.py and puts it in a > PetscBinaryRead.py that reads a binary with (potentially) multiple Petsc > objects, deciphers the contents of the file from the header, and returns > numpy objects. ?It's basically the same as the PetscBinaryRead.m > > I've tested this on Vecs, but not Mats or ISs, and I haven't thoroughly > tested degenerate cases. ?Will do some testing, but I wanted Lisandro to > take a look at it as well... > 1) readMatDense() is wrong. The I,J,V arrays are the CSR structure, not the COO (coordinate) format, Then you cannot simply: for i,j,v in zip(I,J,V): mat[i,j] = v I think you have to: for row, rstart in enumerate(I): rend = I[row+1] mat[row, J[rstart:rend]] = V[rstart:rend] 2) It would be nice to provie a 'scipy' Mat format returning a 'scipy.sparse.csr_matrix' instance. A possible implementation would be: def readMatSciPy(fh): (M, N), (I, J, V) = readMatSparse(fh) from scipy.sparse import csr_matrix return csr_matrix((V, J, I), shape=(M, N)) -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From ecoon at lanl.gov Thu Jul 14 16:33:41 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Thu, 14 Jul 2011 15:33:41 -0600 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <1310671333.22449.180.camel@echo.lanl.gov> Message-ID: <1310679221.22449.198.camel@echo.lanl.gov> On Thu, 2011-07-14 at 17:47 -0300, Lisandro Dalcin wrote: > > 1) readMatDense() is wrong. The I,J,V arrays are the CSR structure, > not the COO (coordinate) format, Then you cannot simply: > > for i,j,v in zip(I,J,V): > mat[i,j] = v > > I think you have to: > > for row, rstart in enumerate(I): > rend = I[row+1] > mat[row, J[rstart:rend]] = V[rstart:rend] > Ok, thanks, I wasn't sure of which format you were using, and didn't have a handy matrix binary file to test it (and didn't want to think too hard about it). I just needed a reader for vecs working today, so was in a hurry to get something done. > > 2) It would be nice to provie a 'scipy' Mat format returning a > 'scipy.sparse.csr_matrix' instance. A possible implementation would > be: > > def readMatSciPy(fh): > (M, N), (I, J, V) = readMatSparse(fh) > from scipy.sparse import csr_matrix > return csr_matrix((V, J, I), shape=(M, N)) > > Agreed, I considered doing that as the default for sparse, but wasn't sure what others would want. Providing both makes sense. I have to finish up some other crap first, but will get these fixes done soon. Note that this should work for people using the release version with no modifications, if Andrew is still reading this. Thanks, Etha > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From dalcinl at gmail.com Thu Jul 14 17:34:42 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 14 Jul 2011 19:34:42 -0300 Subject: [petsc-users] PETSc recommended visualization packages In-Reply-To: <1310679221.22449.198.camel@echo.lanl.gov> References: <600F9A29-6D1B-4004-9FAF-888FED9C24D8@mcs.anl.gov> <1309963270.20331.10.camel@echo.lanl.gov> <1310671333.22449.180.camel@echo.lanl.gov> <1310679221.22449.198.camel@echo.lanl.gov> Message-ID: On 14 July 2011 18:33, Ethan Coon wrote: > On Thu, 2011-07-14 at 17:47 -0300, Lisandro Dalcin wrote: > >> >> 1) readMatDense() is wrong. The I,J,V arrays are the CSR structure, >> not the COO (coordinate) format, Then you cannot simply: >> > > Ok, thanks, I wasn't sure of which format you were using, Well, PETSc's AIJ matrices are CSR in memory (but not in disk, binary files store row counts instead of row pointers) >> >> 2) It would be nice to provie a 'scipy' Mat format returning a >> 'scipy.sparse.csr_matrix' instance. A possible implementation would >> be: >> > > Agreed, I considered doing that as the default for sparse, but wasn't > sure what others would want. ?Providing both makes sense. > Note however that this would require a SciPy installation. PS: Sorry for not being more helpful and contributing actual code, but I'm really busy. -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From rudolph at berkeley.edu Thu Jul 14 19:55:37 2011 From: rudolph at berkeley.edu (Max Rudolph) Date: Thu, 14 Jul 2011 17:55:37 -0700 Subject: [petsc-users] VecPointwiseDivide for Vecs with multiple DOFs? Message-ID: Is there a straightforward way to perform a pointwise divide of every DOF in a global Vec with multiple degrees of freedom by another vector that has only one degree of freedom? Thanks for your help. Max From knepley at gmail.com Thu Jul 14 20:01:43 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 15 Jul 2011 01:01:43 +0000 Subject: [petsc-users] VecPointwiseDivide for Vecs with multiple DOFs? In-Reply-To: References: Message-ID: You would just write this the same way PointwiseDivide() is written. Take out the arrays using VecGetArray() and do the processing yourself. Thanks, Matt On Fri, Jul 15, 2011 at 12:55 AM, Max Rudolph wrote: > Is there a straightforward way to perform a pointwise divide of every DOF > in a global Vec with multiple degrees of freedom by another vector that has > only one degree of freedom? Thanks for your help. > > Max -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nan.Jia at Dartmouth.edu Fri Jul 15 10:00:57 2011 From: Nan.Jia at Dartmouth.edu (Nan.Jia at Dartmouth.edu) Date: Fri, 15 Jul 2011 11:00:57 -0400 Subject: [petsc-users] Can I set up the initial guess for the solution vector for the KSP? Message-ID: <20110715110057.hd98e0wco40swco8@webmail.dartmouth.edu> Dear all, I am trying to solve a time stepping problem, And I'd like to use the solution from the last time step as the initial guess for the iterative solver in KSP,for example, the GMRES method. Can I do it by calling some routines in PETSc? Thanks in advance. Best, Nan From jedbrown at mcs.anl.gov Fri Jul 15 10:18:44 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 15 Jul 2011 10:18:44 -0500 Subject: [petsc-users] Can I set up the initial guess for the solution vector for the KSP? In-Reply-To: <20110715110057.hd98e0wco40swco8@webmail.dartmouth.edu> References: <20110715110057.hd98e0wco40swco8@webmail.dartmouth.edu> Message-ID: On Fri, Jul 15, 2011 at 10:00, wrote: > Dear all, > I am trying to solve a time stepping problem, And I'd like to use the > solution from the last time step as the initial guess for the iterative > solver in KSP,for example, the GMRES method. Can I dit by calling some > routines in PETSc? Thanks in advance. > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/KSP/KSPSetInitialGuessNonzero.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram at ibrae.ac.ru Sat Jul 16 04:55:14 2011 From: ram at ibrae.ac.ru (=?KOI8-R?B?4czFy9PFyiDy0drBzs/X?=) Date: Sat, 16 Jul 2011 13:55:14 +0400 Subject: [petsc-users] tfqmr true residual norm monitoring Message-ID: Hi, Im solving a linear system with different methods for comparison goals. And I want to make Krylov iterative methods work until they reach the predefined value of the *TRUE* residual norm. So Im setting up tolerances (in fact only *atol*) with KSPSetTolerances(KSP, 1.e-50, 1.e-10, PETSC_DEFAULT, PETSC_DEFAULT) and watching for -ksp_monitor (or -ksp_monitor_true_residual). By default most of methods monitor their preconditioned residual norm and stop working then the preconditioned residual norm reaches the predefined value. That of course doesn't mean true residual norm also reached the value. So I need to change this behavior and use KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) to use unpreconditioned residual or KSPSetPreconditionerSide(KSP, PC_RIGHT) to make preconditioned residual be the same as unpreconditioned and achieve my goal anyway. So what I have: 1) KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) is only supported by CG, RICHARDSON AND CHEBYCHEV (a said in user manual). So as it is expected, CG, RICHARDSON AND CHEBYCHEV print error message: "No right preconditioning for KSPCG, KSPRICHARDSON, KSPCHEBYCHEV!" and work great with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) Great! 2) Practically the same behavior does KSPLSQR (except the great work, actually it doesn't work at all with my linear system, but I don't care about lsqr) So it's ok. 3) KSPBCGS and KSPBICG don't support right preconditioning, so their true residual norm values cant be set as the finish point for iterating. Am I right? 4) KSPGMRES print error message: "Use right preconditioning -ksp_right_pc if want unpreconditioned norm", when I use it with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED). And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). As expected. 5) KSPCGS prints no error, when I use it with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) and works with preconditioned norm. And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). So it's ok. 6) KSPTFQMR and KSPTCQMR print no errors with any settings and works with preconditioned norm. Is there any methods to make KSPTFQMR use unpreconditioned norm or right preconditioning? Please help me to understand situations 3) and 6) Thank in advance! -- Best regards, Alexey Ryazanov ______________________________________ Nuclear Safety Institute of Russian Academy of Sciences -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Jul 16 16:06:11 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 16 Jul 2011 16:06:11 -0500 Subject: [petsc-users] tfqmr true residual norm monitoring In-Reply-To: References: Message-ID: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> On Jul 16, 2011, at 4:55 AM, ??????? ??????? wrote: > Hi, Im solving a linear system with different methods for comparison goals. > And I want to make Krylov iterative methods work until they reach the predefined value of the TRUE residual norm. > So Im setting up tolerances (in fact only atol) with KSPSetTolerances(KSP, 1.e-50, 1.e-10, PETSC_DEFAULT, PETSC_DEFAULT) and watching for -ksp_monitor (or -ksp_monitor_true_residual). > By default most of methods monitor their preconditioned residual norm and stop working then the preconditioned residual norm reaches the predefined value. > That of course doesn't mean true residual norm also reached the value. > > > So I need to change this behavior and use KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) to use unpreconditioned residual or KSPSetPreconditionerSide(KSP, PC_RIGHT) to make preconditioned residual be the same as unpreconditioned and achieve my goal anyway. > > So what I have: > > > 1) > KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) is only supported by CG, RICHARDSON AND CHEBYCHEV (a said in user manual). > So as it is expected, CG, RICHARDSON AND CHEBYCHEV print error message: "No right preconditioning for KSPCG, KSPRICHARDSON, KSPCHEBYCHEV!" and work great with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) > Great! > > 2) > Practically the same behavior does KSPLSQR (except the great work, actually it doesn't work at all with my linear system, but I don't care about lsqr) > So it's ok. > > > 3) > KSPBCGS and KSPBICG don't support right preconditioning, so their true residual norm values cant be set as the finish point for iterating. > Am I right? Actually in petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html KSPBCGS does support both left and right preconditioning. KSPBICG is coded only for left preconditioning > > > 4) > KSPGMRES print error message: "Use right preconditioning -ksp_right_pc if want unpreconditioned norm", when I use it with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED). > And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). > As expected. > > 5) > KSPCGS prints no error, when I use it with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) and works with preconditioned norm. > And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). > So it's ok. > > 6) > KSPTFQMR and KSPTCQMR print no errors with any settings and works with preconditioned norm. > Is there any methods to make KSPTFQMR use unpreconditioned norm or right preconditioning? These two algorithms actually do work with both left and right preconditioner but they are different than all the other methods in that: The "residual norm" computed in this algorithm is actually just an upper bound on the actual residual norm. That is for left preconditioning it is a bound on the preconditioned residual and for right preconditioning it is a bound on the true residual. I have added this information to their manual pages in petsc-dev for clarity. Note that our GMRES is implemented only with left preconditioning and preconditioned residual norm but you can use the FGMRES which is implemented only for right preconditioning and the true residual norm for testing. If you not particularly interested in speed and just want the iterations to stop when the true residual norm reaches some tolerance you can easily provide your own KSPConvergedTest() by calling KSPSetConvergenceTest() and writing a routine that calls KSPBuildResidual() (which always builds the true residual) and compute the norm of the result. Barry > > > Please help me to understand situations 3) and 6) > > Thank in advance! > > -- > Best regards, > Alexey Ryazanov > ______________________________________ > Nuclear Safety Institute of Russian Academy of Sciences > > From jedbrown at mcs.anl.gov Sat Jul 16 16:10:13 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 16 Jul 2011 16:10:13 -0500 Subject: [petsc-users] tfqmr true residual norm monitoring In-Reply-To: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> References: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> Message-ID: On Sat, Jul 16, 2011 at 16:06, Barry Smith wrote: > Note that our GMRES is implemented only with left preconditioning and > preconditioned residual norm but you can use the FGMRES which is implemented > only for right preconditioning and the true residual norm for testing. Right preconditioning also works with normal GMRES, run with -ksp_norm_type unpreconditioned -ksp_pc_side right (in petsc-dev, -ksp_right_pc in petsc-3.1). -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Jul 16 16:12:03 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 16 Jul 2011 16:12:03 -0500 Subject: [petsc-users] tfqmr true residual norm monitoring In-Reply-To: References: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> Message-ID: <7BBBD201-9E37-46E4-ABBA-E47D626CBF6E@mcs.anl.gov> On Jul 16, 2011, at 4:10 PM, Jed Brown wrote: > On Sat, Jul 16, 2011 at 16:06, Barry Smith wrote: > Note that our GMRES is implemented only with left preconditioning and preconditioned residual norm but you can use the FGMRES which is implemented only for right preconditioning and the true residual norm for testing. > > Right preconditioning also works with normal GMRES, run with -ksp_norm_type unpreconditioned -ksp_pc_side right (in petsc-dev, -ksp_right_pc in petsc-3.1). Oh yes. I keep forgetting this. Barry From amryazanov at gmail.com Sat Jul 16 16:58:30 2011 From: amryazanov at gmail.com (=?KOI8-R?B?4czFy9PFyiDy0drBzs/X?=) Date: Sun, 17 Jul 2011 01:58:30 +0400 Subject: [petsc-users] tfqmr true residual norm monitoring In-Reply-To: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> References: <8A8FC4CB-3547-4275-8871-468DB53D5AD7@mcs.anl.gov> Message-ID: Thank You very much! I don't know Why I was avoiding the ownprecond variant. May be just was scared to try. I think I'll use it. I also could just set the maximum number of iterations and watch the -ksp_monitor_true_residual, but it's not so convinient. Thank You! 2011/7/17 Barry Smith > > On Jul 16, 2011, at 4:55 AM, ??????? ??????? wrote: > > > Hi, Im solving a linear system with different methods for comparison > goals. > > And I want to make Krylov iterative methods work until they reach the > predefined value of the TRUE residual norm. > > So Im setting up tolerances (in fact only atol) with > KSPSetTolerances(KSP, 1.e-50, 1.e-10, PETSC_DEFAULT, PETSC_DEFAULT) and > watching for -ksp_monitor (or -ksp_monitor_true_residual). > > By default most of methods monitor their preconditioned residual norm and > stop working then the preconditioned residual norm reaches the predefined > value. > > That of course doesn't mean true residual norm also reached the value. > > > > > > So I need to change this behavior and use KSPSetNormType(KSP, > KSP_NORM_UNPRECONDITIONED) to use unpreconditioned residual or > KSPSetPreconditionerSide(KSP, PC_RIGHT) to make preconditioned residual be > the same as unpreconditioned and achieve my goal anyway. > > > > So what I have: > > > > > > 1) > > KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) is only supported by CG, > RICHARDSON AND CHEBYCHEV (a said in user manual). > > So as it is expected, CG, RICHARDSON AND CHEBYCHEV print error message: > "No right preconditioning for KSPCG, KSPRICHARDSON, KSPCHEBYCHEV!" and work > great with KSPSetNormType(KSP, KSP_NORM_UNPRECONDITIONED) > > Great! > > > > 2) > > Practically the same behavior does KSPLSQR (except the great work, > actually it doesn't work at all with my linear system, but I don't care > about lsqr) > > So it's ok. > > > > > > 3) > > KSPBCGS and KSPBICG don't support right preconditioning, so their true > residual norm values cant be set as the finish point for iterating. > > Am I right? > > Actually in petsc-dev > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html KSPBCGS does > support both left and right preconditioning. KSPBICG is coded only for left > preconditioning > > > > > > 4) > > KSPGMRES print error message: "Use right preconditioning -ksp_right_pc if > want unpreconditioned norm", when I use it with KSPSetNormType(KSP, > KSP_NORM_UNPRECONDITIONED). > > And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). > > As expected. > > > > 5) > > KSPCGS prints no error, when I use it with KSPSetNormType(KSP, > KSP_NORM_UNPRECONDITIONED) and works with preconditioned norm. > > And works great with KSPSetPreconditionerSide(KSP, PC_RIGHT). > > So it's ok. > > > > 6) > > KSPTFQMR and KSPTCQMR print no errors with any settings and works with > preconditioned norm. > > Is there any methods to make KSPTFQMR use unpreconditioned norm or right > preconditioning? > > These two algorithms actually do work with both left and right > preconditioner but they are different than all the other methods in that: > > The "residual norm" computed in this algorithm is actually just an upper > bound on the actual residual norm. > That is for left preconditioning it is a bound on the > preconditioned residual and for right preconditioning > it is a bound on the true residual. > > I have added this information to their manual pages in petsc-dev for > clarity. > > Note that our GMRES is implemented only with left preconditioning and > preconditioned residual norm but you can use the FGMRES which is implemented > only for right preconditioning and the true residual norm for testing. > > > If you not particularly interested in speed and just want the iterations > to stop when the true residual norm reaches some tolerance you can easily > provide your own KSPConvergedTest() by calling KSPSetConvergenceTest() and > writing a routine that calls KSPBuildResidual() (which always builds the > true residual) and compute the norm of the result. > > Barry > > > > > > > > Please help me to understand situations 3) and 6) > > > > Thank in advance! > > > > -- > > Best regards, > > Alexey Ryazanov > > ______________________________________ > > Nuclear Safety Institute of Russian Academy of Sciences > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Sun Jul 17 04:44:45 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sun, 17 Jul 2011 17:44:45 +0800 (CST) Subject: [petsc-users] superlu ILUT preconditioner break Message-ID: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> Hi, Several months ago, I posted message "superlu ILUT preconditioner break" with following link http://lists.mcs.anl.gov/pipermail/petsc-users/2011-March/008355.html Today I studied this problem carefully to see why it crashes. The segment fault is arising at line 692 of superlu's dgsisx.c, because the array "perm" is 0. This array which contains the permutation calculated from MC64 procedure, can only be initialized when nofact flag is true. However, petsc will call dgsisx twice, with nofact value 1 and 0, respectively. In the second calling, the perm array is not initialized but be used. That's why the code crash. Hope this bug can be fixed without too many efforts. Gong Ding From gdiso at ustc.edu Sun Jul 17 04:53:40 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sun, 17 Jul 2011 17:53:40 +0800 (CST) Subject: [petsc-users] superlu ILUT preconditioner break In-Reply-To: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> References: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> Message-ID: <7560486.130791310896420502.JavaMail.coremail@mail.ustc.edu> BTW, this problem can be avoided by setting -mat_superlu_rowperm to NOROWPERM instead of default value LargeDiag > > Several months ago, I posted message "superlu ILUT preconditioner break" with following link > > http://lists.mcs.anl.gov/pipermail/petsc-users/2011-March/008355.html > > > > Today I studied this problem carefully to see why it crashes. > > > > The segment fault is arising at line 692 of superlu's dgsisx.c, because the array "perm" is 0. > > This array which contains the permutation calculated from MC64 procedure, can only be initialized when nofact flag is true. However, petsc will call dgsisx twice, with nofact value 1 and 0, respectively. In the second calling, the perm array is not initialized but be used. That's why the code crash. > > > > Hope this bug can be fixed without too many efforts. > > > > Gong Ding > > > > > > > > > > > > From hzhang at mcs.anl.gov Sun Jul 17 10:10:35 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Sun, 17 Jul 2011 10:10:35 -0500 Subject: [petsc-users] superlu ILUT preconditioner break In-Reply-To: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> References: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> Message-ID: Gong Ding : You found the exact reason, which we have reported to the superlu team earlier this year. See Sherry's response attached below. Hong --------------------------------------- from Xiaoye S. Li xsli at lbl.gov to Hong Zhang cc Meiyue Shao date Tue, Feb 22, 2011 at 4:46 PM subject Re: superlu ilu hide details Feb 22 Hong, Thank you very much for the providing the detailed diagnostics. Apparently, we didn't test the case when dgsisx() is called twice with separate factorization and solve. The best solution is to merge this "perm" (row permuation from MC64, obtained before factorization)) into "perm_r" (row permutation from partial pivoting duing factorization). In the end, the routine will return both permutations in a fused "perm_r". Meiyue -- Do you agree with this fix? I will need some time to test this out, and let you know once it's ready. Sherry > > Several months ago, I posted message "superlu ILUT preconditioner break" with following link > http://lists.mcs.anl.gov/pipermail/petsc-users/2011-March/008355.html > > Today I studied this problem carefully to see why it crashes. > > The segment fault is arising at line 692 of superlu's dgsisx.c, because the array "perm" is 0. > This array which contains the permutation calculated from MC64 procedure, can only be initialized when nofact flag is true. However, petsc will call dgsisx twice, with nofact value 1 and 0, respectively. ?In the second calling, the perm array is not initialized but be used. That's why the code crash. > > Hope this bug can be fixed without too many efforts. > > Gong Ding > > > > > > From hzhang at mcs.anl.gov Sun Jul 17 10:16:29 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Sun, 17 Jul 2011 10:16:29 -0500 Subject: [petsc-users] superlu ILUT preconditioner break In-Reply-To: <7560486.130791310896420502.JavaMail.coremail@mail.ustc.edu> References: <25439332.130761310895885154.JavaMail.coremail@mail.ustc.edu> <7560486.130791310896420502.JavaMail.coremail@mail.ustc.edu> Message-ID: We have set options.RowPerm = NOROWPERM; in petsc-dev/superlu interface. see petsc-dev/src/mat/impls/aij/seq/superlu/superlu.c Are you using the latest petsc-dev? Hong 2011/7/17 Gong Ding : > BTW, this problem can be avoided by setting > -mat_superlu_rowperm to NOROWPERM instead of default value LargeDiag > > >> >> Several months ago, I posted message "superlu ILUT preconditioner break" with following link >> >> http://lists.mcs.anl.gov/pipermail/petsc-users/2011-March/008355.html >> >> >> >> Today I studied this problem carefully to see why it crashes. >> >> >> >> The segment fault is arising at line 692 of superlu's dgsisx.c, because the array "perm" is 0. >> >> This array which contains the permutation calculated from MC64 procedure, can only be initialized when nofact flag is true. However, petsc will call dgsisx twice, with nofact value 1 and 0, respectively. ?In the second calling, the perm array is not initialized but be used. That's why the code crash. >> >> >> >> Hope this bug can be fixed without too many efforts. >> >> >> >> Gong Ding >> >> >> >> >> >> >> >> >> >> >> >> > From clemens.domanig at uibk.ac.at Mon Jul 18 06:46:03 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Mon, 18 Jul 2011 13:46:03 +0200 Subject: [petsc-users] How to get diagonal entries ... Message-ID: <4E241CFB.5080000@uibk.ac.at> Hi everyone, I did a Cholesky-decomposition using MUMPS and now I want to get the diagonal entries of the Cholesky-decomposition but MatGetDiagonal doesnt't seem to work. Thanks for your help - Clemens Domanig ... ierr=PCSetType( prec, PCCHOLESKY); CHKERRQ( ierr); ... ierr=PCFactorGetMatrix( prec, &M); CHKERRQ( ierr); ierr=MatGetDiagonal( M, z); CHKERRQ( ierr); printf("\nPivots:\n\n"); ierr=VecView( z, PETSC_VIEWER_STDOUT_SELF); CHKERRQ( ierr); output Process [0] inf 0 0 From gianmail at gmail.com Mon Jul 18 09:43:17 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Mon, 18 Jul 2011 16:43:17 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> Message-ID: Barry, thanks again for your input. That would be really good. When executing my program with the options you suggested I obtain various errors (unknown pc etc) so I tried to figure out how to avoid them. Is it something like this what you intended? ./main -mg_pc_type composite -mg_pc_composite_pcs ilu,ilu -mg_ksp_type preonly -mg_sub_0_pc_factor_mat_ordering_type FESOrder -mg_sub_1_pc_factor_mat_ordering_type FWSOrder -mg_pc_composite_type multiplicative -mg_ksp_view provided that KSPSetOptionsPrefix(ksp,"mg_"); has been applied to the KSP I want to modify. Thanks again Gianluca On 13 July 2011 15:33, Barry Smith wrote: > > On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: > >> Thanks Barry. >> >> What I'm doing at the moment in order to pass the ordering to the PC is >> >> PCFactorSetMatOrderingType(PC,"FESOrder"); >> >> where the FESOrder(...) function computes the ordering and has been >> registered in the main function with >> >> MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); >> >> The same is done for al the different orderings I'm using inside the >> PCSHELL suggested by Jed. >> >> My understanding is that the FESOrder(...) function is evaluated every >> time the PCApply(...) function is called. > > ? Definitely not. Looking at PCSetUp_ILU() > > ?} else { > ? ?if (!pc->setupcalled) { > ? ? ?/* first time in so compute reordering and symbolic factorization */ > ? ? ?ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); > > it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). > > ? BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. ?Just create a single KSP in the usual way and use the options > > -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse > > > ? Barry > > > > > >> >> If there was a way to store the IS containing the ordering ? or any >> other array ? into the shell context (this I know how to do that) and >> the pass the IS directly to the preconditioner instead of using >> PCFactorSetMatOrderingType would be the best solution for me. >> >> Thanks >> >> Gianluca >> >> >> On 12 July 2011 20:15, Barry Smith wrote: >>> >>> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: >>> >>>> Jed, thanks for your answer. >>>> >>>> This is exactly what I was looking for (it took me some time to test it...). >>>> >>>> Let me ask you a couple of additional questions: >>>> >>>> 1) in this configuration, is there an alternative way to pass the >>>> ordering to the preconditioner? Becuse my ordering depends on the grid >>>> an not on the solution, it is constant during all the computations and >>>> I have no advantage on calling the function that creates multiple >>>> times (the pc is built and destroyed several times). >>> >>> ? Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. >>> >>> ? ?PCShellSet/GetContext()? >>>> >>>> 2) what's the best way to make the shell pc selectable from the option >>>> database, so that it can be replaced with more standard ones? >>>> >>> >>> ? ? ? ?/* here the user can provide -pc_type shell or ilu or whatever */ >>> ? ? ? ?KSPSetFromOptions(ksp); >>> ? ? ? ?KSPGetPC(ksp,&pc); >>> ? ? ? ?/* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ >>> >>> ? ? ? ?PCShellSetContext(pc,....) >>> ? ? ? ?PCShellSetSetUp(pc,....) >>> ? ? ? ?PCShellSetApply(pc,.....) >>> >>> >>> ? Barry >>> >>> >>>> Thanks >>>> >>>> Gianluca >>>> >>>> On 1 July 2011 14:43, Jed Brown wrote: >>>>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>>>> wrote: >>>>>> >>>>>> I've tried to implement Jed's solution as it was looking faster to >>>>>> try. This is what I came up with for two different orders, FENOrder >>>>>> and FWNOrder respectively >>>>>> >>>>>> 283 ? ? ? KSP ksp; >>>>>> 284 ? ? ? PC pc; >>>>>> 285 >>>>>> 286 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >>>>>> 287 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >>>>>> 288 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>> CHKERRQ(ierr); >>>>>> 289 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >>>>>> 290 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >>>>>> 291 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >>>>>> 292 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>>>> 293 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>> 294 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); >>>>>> 295 >>>>>> 296 ? ? ? ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); ?CHKERRQ(ierr); >>>>>> 297 ? ? ? ierr = KSPSetOptionsPrefix(ksp,"mg_"); ? ?CHKERRQ(ierr); >>>>>> 298 ? ? ? ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>> CHKERRQ(ierr); >>>>>> 299 ? ? ? ierr = KSPSetFromOptions(ksp); ? ?CHKERRQ(ierr); >>>>>> 300 ? ? ? ierr = KSPGetPC(ksp,&pc); ? ? ? ? CHKERRQ(ierr); >>>>>> 301 ? ? ? ierr = PCFactorSetFill(pc,1); ? ? CHKERRQ(ierr); >>>>>> 302 ? ? ? ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>>>> 303 ? ? ? ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>> 304 ? ? ? ierr = KSPDestroy(&ksp); ?CHKERRQ(ierr); >>>>> >>>>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>>>> making >>>>> struct YourShellContext { >>>>> ? PC fenpc,fwnpc; >>>>> ? Vec work; >>>>> ? PetscBool issetup; >>>>> }; >>>>> Then pass a pointer to this thing in when you create the PCShell. Then >>>>> something like >>>>> PCSetUp_YourShell(PC pc) { >>>>> struct YourShellContext *ctx; >>>>> Mat A,B; >>>>> MatStructure mstr; >>>>> const char *prefix; >>>>> char iprefix[256]; >>>>> PCShellGetContext(pc,&ctx); >>>>> PCGetOptionsPrefix(pc,&prefix); >>>>> if (!ctx->issetup) { >>>>> ? ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>>>> ? PCCreate(comm,&ctx->fenpc); >>>>> ? PCSetType(ctx->fenpc,PCILU); >>>>> ? PCFactorSetFill(ctx->fenpc,1); >>>>> ? PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>>>> ? snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>>>> ? PCSetOptionsPrefix(ctx->fenpc,iprefix); >>>>> ? PCSetFromOptions(ctx->fenpc); >>>>> ? /* Same as above for ctx->fwnpc */ >>>>> } >>>>> PCGetOperators(pc,&A,&B,&mstr); >>>>> PCSetOperators(ctx->fenpc,A,B,mstr); >>>>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>>>> PCSetUp(ctx->fenpc); >>>>> PCSetUp(ctx->fwnpc); >>>>> ctx->issetup = PETSC_TRUE; >>>>> } >>>>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>>>> ... >>>>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>>>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>>>> VecScale(Y,-1.0); >>>>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>>>> PCApply(ctx->fwnpc,ctx->work,Y); >>>>> } >>>>>> >>>>>> I don't know if this is what you intended... is there a way to make >>>>>> KSP recompute the factorization with the different ordering without >>>>>> destroying and recreating the KSP? >>>>> >>>>> Store both as in the code above. >>>>> >>>>>> >>>>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>>>> factorization function (and eventually the ones provided by external >>>>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>>>> need? If so, what's the best way to access it? >>>>> >>>>> Does the code above answer this question? >>>> >>>> >>>> >>>> -- >>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>> Amedeo Modigliani >>> >>> >> >> >> >> -- >> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >> l'Anonymat" -- Non omnibus, sed mihi et tibi >> Amedeo Modigliani > > -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From hzhang at mcs.anl.gov Mon Jul 18 09:59:56 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Mon, 18 Jul 2011 09:59:56 -0500 Subject: [petsc-users] How to get diagonal entries ... In-Reply-To: <4E241CFB.5080000@uibk.ac.at> References: <4E241CFB.5080000@uibk.ac.at> Message-ID: Clemens: > > I did a Cholesky-decomposition using MUMPS and now I want to get the > diagonal entries of the Cholesky-decomposition but MatGetDiagonal doesnt't > seem to work. MUMPS uses its internal data structure for matrix factorization and does not provides user interface to access it, thus Petsc has noway to do is as well. Try option '-mat_mumps_icntl_4 4' (level of printing) to see if you can get useful info from mumps. Other wise, send request to mumps developer. Hong > > Thanks for your help - Clemens Domanig > > ... > ierr=PCSetType( prec, PCCHOLESKY); CHKERRQ( ierr); > ... > ierr=PCFactorGetMatrix( prec, &M); CHKERRQ( ierr); > ierr=MatGetDiagonal( M, z); CHKERRQ( ierr); > printf("\nPivots:\n\n"); > ierr=VecView( z, PETSC_VIEWER_STDOUT_SELF); CHKERRQ( ierr); > > output > Process [0] > inf > 0 > 0 > From bsmith at mcs.anl.gov Mon Jul 18 13:14:08 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 18 Jul 2011 13:14:08 -0500 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> Message-ID: On Jul 18, 2011, at 9:43 AM, Gianluca Meneghello wrote: > Barry, > > thanks again for your input. That would be really good. > > When executing my program with the options you suggested I obtain > various errors (unknown pc etc) so I tried to figure out how to avoid > them. Is it something like this what you intended? > > ./main > -mg_pc_type composite > -mg_pc_composite_pcs ilu,ilu > -mg_ksp_type preonly I don't think you want this one. It says not to use a Krylov method on the outside; the preconditioner will just apply the two ILU triangular and that is it. In other words it won't even solve the linear system. You likely want -mg_ksp_type gmres Barry > -mg_sub_0_pc_factor_mat_ordering_type FESOrder > -mg_sub_1_pc_factor_mat_ordering_type FWSOrder > -mg_pc_composite_type multiplicative > -mg_ksp_view > > provided that > > KSPSetOptionsPrefix(ksp,"mg_"); > > has been applied to the KSP I want to modify. > > Thanks again > > Gianluca > > > On 13 July 2011 15:33, Barry Smith wrote: >> >> On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: >> >>> Thanks Barry. >>> >>> What I'm doing at the moment in order to pass the ordering to the PC is >>> >>> PCFactorSetMatOrderingType(PC,"FESOrder"); >>> >>> where the FESOrder(...) function computes the ordering and has been >>> registered in the main function with >>> >>> MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); >>> >>> The same is done for al the different orderings I'm using inside the >>> PCSHELL suggested by Jed. >>> >>> My understanding is that the FESOrder(...) function is evaluated every >>> time the PCApply(...) function is called. >> >> Definitely not. Looking at PCSetUp_ILU() >> >> } else { >> if (!pc->setupcalled) { >> /* first time in so compute reordering and symbolic factorization */ >> ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); >> >> it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). >> >> BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. Just create a single KSP in the usual way and use the options >> >> -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse >> >> >> Barry >> >> >> >> >> >>> >>> If there was a way to store the IS containing the ordering ? or any >>> other array ? into the shell context (this I know how to do that) and >>> the pass the IS directly to the preconditioner instead of using >>> PCFactorSetMatOrderingType would be the best solution for me. >>> >>> Thanks >>> >>> Gianluca >>> >>> >>> On 12 July 2011 20:15, Barry Smith wrote: >>>> >>>> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: >>>> >>>>> Jed, thanks for your answer. >>>>> >>>>> This is exactly what I was looking for (it took me some time to test it...). >>>>> >>>>> Let me ask you a couple of additional questions: >>>>> >>>>> 1) in this configuration, is there an alternative way to pass the >>>>> ordering to the preconditioner? Becuse my ordering depends on the grid >>>>> an not on the solution, it is constant during all the computations and >>>>> I have no advantage on calling the function that creates multiple >>>>> times (the pc is built and destroyed several times). >>>> >>>> Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. >>>> >>>> PCShellSet/GetContext()? >>>>> >>>>> 2) what's the best way to make the shell pc selectable from the option >>>>> database, so that it can be replaced with more standard ones? >>>>> >>>> >>>> /* here the user can provide -pc_type shell or ilu or whatever */ >>>> KSPSetFromOptions(ksp); >>>> KSPGetPC(ksp,&pc); >>>> /* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ >>>> >>>> PCShellSetContext(pc,....) >>>> PCShellSetSetUp(pc,....) >>>> PCShellSetApply(pc,.....) >>>> >>>> >>>> Barry >>>> >>>> >>>>> Thanks >>>>> >>>>> Gianluca >>>>> >>>>> On 1 July 2011 14:43, Jed Brown wrote: >>>>>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>>>>> wrote: >>>>>>> >>>>>>> I've tried to implement Jed's solution as it was looking faster to >>>>>>> try. This is what I came up with for two different orders, FENOrder >>>>>>> and FWNOrder respectively >>>>>>> >>>>>>> 283 KSP ksp; >>>>>>> 284 PC pc; >>>>>>> 285 >>>>>>> 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>> 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>> 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>> CHKERRQ(ierr); >>>>>>> 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>> 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>> 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>> 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>>>>> 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>> 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>>> 295 >>>>>>> 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>> 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>> 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>> CHKERRQ(ierr); >>>>>>> 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>> 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>> 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>> 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>>>>> 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>> 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>> >>>>>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>>>>> making >>>>>> struct YourShellContext { >>>>>> PC fenpc,fwnpc; >>>>>> Vec work; >>>>>> PetscBool issetup; >>>>>> }; >>>>>> Then pass a pointer to this thing in when you create the PCShell. Then >>>>>> something like >>>>>> PCSetUp_YourShell(PC pc) { >>>>>> struct YourShellContext *ctx; >>>>>> Mat A,B; >>>>>> MatStructure mstr; >>>>>> const char *prefix; >>>>>> char iprefix[256]; >>>>>> PCShellGetContext(pc,&ctx); >>>>>> PCGetOptionsPrefix(pc,&prefix); >>>>>> if (!ctx->issetup) { >>>>>> ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>>>>> PCCreate(comm,&ctx->fenpc); >>>>>> PCSetType(ctx->fenpc,PCILU); >>>>>> PCFactorSetFill(ctx->fenpc,1); >>>>>> PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>>>>> snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>>>>> PCSetOptionsPrefix(ctx->fenpc,iprefix); >>>>>> PCSetFromOptions(ctx->fenpc); >>>>>> /* Same as above for ctx->fwnpc */ >>>>>> } >>>>>> PCGetOperators(pc,&A,&B,&mstr); >>>>>> PCSetOperators(ctx->fenpc,A,B,mstr); >>>>>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>>>>> PCSetUp(ctx->fenpc); >>>>>> PCSetUp(ctx->fwnpc); >>>>>> ctx->issetup = PETSC_TRUE; >>>>>> } >>>>>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>>>>> ... >>>>>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>>>>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>>>>> VecScale(Y,-1.0); >>>>>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>>>>> PCApply(ctx->fwnpc,ctx->work,Y); >>>>>> } >>>>>>> >>>>>>> I don't know if this is what you intended... is there a way to make >>>>>>> KSP recompute the factorization with the different ordering without >>>>>>> destroying and recreating the KSP? >>>>>> >>>>>> Store both as in the code above. >>>>>> >>>>>>> >>>>>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>>>>> factorization function (and eventually the ones provided by external >>>>>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>>>>> need? If so, what's the best way to access it? >>>>>> >>>>>> Does the code above answer this question? >>>>> >>>>> >>>>> >>>>> -- >>>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>>> Amedeo Modigliani >>>> >>>> >>> >>> >>> >>> -- >>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>> Amedeo Modigliani >> >> > > > > -- > "[Je pense que] l'homme est un monde qui vaut des fois les mondes et > que les plus ardentes ambitions sont celles qui ont eu l'orgueil de > l'Anonymat" -- Non omnibus, sed mihi et tibi > Amedeo Modigliani From gianmail at gmail.com Mon Jul 18 13:36:36 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Mon, 18 Jul 2011 20:36:36 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> Message-ID: <48F71658-069D-4C41-8294-E55A53CA6748@gmail.com> Barry, I'm actually using ilu as a smoother inside a multigrid cycle. Does in this case make sense to use ksppreonly? Or should I use Richardson? Thanks Il giorno 18/lug/2011, alle ore 20:14, Barry Smith ha scritto: > > On Jul 18, 2011, at 9:43 AM, Gianluca Meneghello wrote: > >> Barry, >> >> thanks again for your input. That would be really good. >> >> When executing my program with the options you suggested I obtain >> various errors (unknown pc etc) so I tried to figure out how to avoid >> them. Is it something like this what you intended? >> >> ./main >> -mg_pc_type composite >> -mg_pc_composite_pcs ilu,ilu >> -mg_ksp_type preonly > > I don't think you want this one. It says not to use a Krylov method on the outside; the preconditioner will just apply the two ILU triangular and that is it. In other words it won't even solve the linear system. You likely want -mg_ksp_type gmres > > Barry > >> -mg_sub_0_pc_factor_mat_ordering_type FESOrder >> -mg_sub_1_pc_factor_mat_ordering_type FWSOrder >> -mg_pc_composite_type multiplicative >> -mg_ksp_view >> >> provided that >> >> KSPSetOptionsPrefix(ksp,"mg_"); >> >> has been applied to the KSP I want to modify. >> >> Thanks again >> >> Gianluca >> >> >> On 13 July 2011 15:33, Barry Smith wrote: >>> >>> On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: >>> >>>> Thanks Barry. >>>> >>>> What I'm doing at the moment in order to pass the ordering to the PC is >>>> >>>> PCFactorSetMatOrderingType(PC,"FESOrder"); >>>> >>>> where the FESOrder(...) function computes the ordering and has been >>>> registered in the main function with >>>> >>>> MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); >>>> >>>> The same is done for al the different orderings I'm using inside the >>>> PCSHELL suggested by Jed. >>>> >>>> My understanding is that the FESOrder(...) function is evaluated every >>>> time the PCApply(...) function is called. >>> >>> Definitely not. Looking at PCSetUp_ILU() >>> >>> } else { >>> if (!pc->setupcalled) { >>> /* first time in so compute reordering and symbolic factorization */ >>> ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); >>> >>> it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). >>> >>> BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. Just create a single KSP in the usual way and use the options >>> >>> -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse >>> >>> >>> Barry >>> >>> >>> >>> >>> >>>> >>>> If there was a way to store the IS containing the ordering ? or any >>>> other array ? into the shell context (this I know how to do that) and >>>> the pass the IS directly to the preconditioner instead of using >>>> PCFactorSetMatOrderingType would be the best solution for me. >>>> >>>> Thanks >>>> >>>> Gianluca >>>> >>>> >>>> On 12 July 2011 20:15, Barry Smith wrote: >>>>> >>>>> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: >>>>> >>>>>> Jed, thanks for your answer. >>>>>> >>>>>> This is exactly what I was looking for (it took me some time to test it...). >>>>>> >>>>>> Let me ask you a couple of additional questions: >>>>>> >>>>>> 1) in this configuration, is there an alternative way to pass the >>>>>> ordering to the preconditioner? Becuse my ordering depends on the grid >>>>>> an not on the solution, it is constant during all the computations and >>>>>> I have no advantage on calling the function that creates multiple >>>>>> times (the pc is built and destroyed several times). >>>>> >>>>> Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. >>>>> >>>>> PCShellSet/GetContext()? >>>>>> >>>>>> 2) what's the best way to make the shell pc selectable from the option >>>>>> database, so that it can be replaced with more standard ones? >>>>>> >>>>> >>>>> /* here the user can provide -pc_type shell or ilu or whatever */ >>>>> KSPSetFromOptions(ksp); >>>>> KSPGetPC(ksp,&pc); >>>>> /* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ >>>>> >>>>> PCShellSetContext(pc,....) >>>>> PCShellSetSetUp(pc,....) >>>>> PCShellSetApply(pc,.....) >>>>> >>>>> >>>>> Barry >>>>> >>>>> >>>>>> Thanks >>>>>> >>>>>> Gianluca >>>>>> >>>>>> On 1 July 2011 14:43, Jed Brown wrote: >>>>>>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>>>>>> wrote: >>>>>>>> >>>>>>>> I've tried to implement Jed's solution as it was looking faster to >>>>>>>> try. This is what I came up with for two different orders, FENOrder >>>>>>>> and FWNOrder respectively >>>>>>>> >>>>>>>> 283 KSP ksp; >>>>>>>> 284 PC pc; >>>>>>>> 285 >>>>>>>> 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>>> 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>>> 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>>> CHKERRQ(ierr); >>>>>>>> 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>>> 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>>> 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>>> 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>>>>>> 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>>> 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>>>> 295 >>>>>>>> 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>>> 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>>> 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>>> CHKERRQ(ierr); >>>>>>>> 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>>> 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>>> 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>>> 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>>>>>> 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>>> 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>>> >>>>>>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>>>>>> making >>>>>>> struct YourShellContext { >>>>>>> PC fenpc,fwnpc; >>>>>>> Vec work; >>>>>>> PetscBool issetup; >>>>>>> }; >>>>>>> Then pass a pointer to this thing in when you create the PCShell. Then >>>>>>> something like >>>>>>> PCSetUp_YourShell(PC pc) { >>>>>>> struct YourShellContext *ctx; >>>>>>> Mat A,B; >>>>>>> MatStructure mstr; >>>>>>> const char *prefix; >>>>>>> char iprefix[256]; >>>>>>> PCShellGetContext(pc,&ctx); >>>>>>> PCGetOptionsPrefix(pc,&prefix); >>>>>>> if (!ctx->issetup) { >>>>>>> ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>>>>>> PCCreate(comm,&ctx->fenpc); >>>>>>> PCSetType(ctx->fenpc,PCILU); >>>>>>> PCFactorSetFill(ctx->fenpc,1); >>>>>>> PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>>>>>> snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>>>>>> PCSetOptionsPrefix(ctx->fenpc,iprefix); >>>>>>> PCSetFromOptions(ctx->fenpc); >>>>>>> /* Same as above for ctx->fwnpc */ >>>>>>> } >>>>>>> PCGetOperators(pc,&A,&B,&mstr); >>>>>>> PCSetOperators(ctx->fenpc,A,B,mstr); >>>>>>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>>>>>> PCSetUp(ctx->fenpc); >>>>>>> PCSetUp(ctx->fwnpc); >>>>>>> ctx->issetup = PETSC_TRUE; >>>>>>> } >>>>>>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>>>>>> ... >>>>>>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>>>>>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>>>>>> VecScale(Y,-1.0); >>>>>>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>>>>>> PCApply(ctx->fwnpc,ctx->work,Y); >>>>>>> } >>>>>>>> >>>>>>>> I don't know if this is what you intended... is there a way to make >>>>>>>> KSP recompute the factorization with the different ordering without >>>>>>>> destroying and recreating the KSP? >>>>>>> >>>>>>> Store both as in the code above. >>>>>>> >>>>>>>> >>>>>>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>>>>>> factorization function (and eventually the ones provided by external >>>>>>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>>>>>> need? If so, what's the best way to access it? >>>>>>> >>>>>>> Does the code above answer this question? >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>>>> Amedeo Modigliani >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>> Amedeo Modigliani >>> >>> >> >> >> >> -- >> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >> l'Anonymat" -- Non omnibus, sed mihi et tibi >> Amedeo Modigliani > From bsmith at mcs.anl.gov Mon Jul 18 13:41:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 18 Jul 2011 13:41:00 -0500 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: <48F71658-069D-4C41-8294-E55A53CA6748@gmail.com> References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> <48F71658-069D-4C41-8294-E55A53CA6748@gmail.com> Message-ID: <8A766A6C-A0A6-440B-974D-1338FC6424E4@mcs.anl.gov> richardson, you may also need to set that KSP with KSPSetInitialGuessNonzero() Barry On Jul 18, 2011, at 1:36 PM, Gianluca Meneghello wrote: > Barry, > I'm actually using ilu as a smoother inside a multigrid cycle. Does in this case make sense to use ksppreonly? Or should I use Richardson? > > Thanks > > > > Il giorno 18/lug/2011, alle ore 20:14, Barry Smith ha scritto: > >> >> On Jul 18, 2011, at 9:43 AM, Gianluca Meneghello wrote: >> >>> Barry, >>> >>> thanks again for your input. That would be really good. >>> >>> When executing my program with the options you suggested I obtain >>> various errors (unknown pc etc) so I tried to figure out how to avoid >>> them. Is it something like this what you intended? >>> >>> ./main >>> -mg_pc_type composite >>> -mg_pc_composite_pcs ilu,ilu >>> -mg_ksp_type preonly >> >> I don't think you want this one. It says not to use a Krylov method on the outside; the preconditioner will just apply the two ILU triangular and that is it. In other words it won't even solve the linear system. You likely want -mg_ksp_type gmres >> >> Barry >> >>> -mg_sub_0_pc_factor_mat_ordering_type FESOrder >>> -mg_sub_1_pc_factor_mat_ordering_type FWSOrder >>> -mg_pc_composite_type multiplicative >>> -mg_ksp_view >>> >>> provided that >>> >>> KSPSetOptionsPrefix(ksp,"mg_"); >>> >>> has been applied to the KSP I want to modify. >>> >>> Thanks again >>> >>> Gianluca >>> >>> >>> On 13 July 2011 15:33, Barry Smith wrote: >>>> >>>> On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: >>>> >>>>> Thanks Barry. >>>>> >>>>> What I'm doing at the moment in order to pass the ordering to the PC is >>>>> >>>>> PCFactorSetMatOrderingType(PC,"FESOrder"); >>>>> >>>>> where the FESOrder(...) function computes the ordering and has been >>>>> registered in the main function with >>>>> >>>>> MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); >>>>> >>>>> The same is done for al the different orderings I'm using inside the >>>>> PCSHELL suggested by Jed. >>>>> >>>>> My understanding is that the FESOrder(...) function is evaluated every >>>>> time the PCApply(...) function is called. >>>> >>>> Definitely not. Looking at PCSetUp_ILU() >>>> >>>> } else { >>>> if (!pc->setupcalled) { >>>> /* first time in so compute reordering and symbolic factorization */ >>>> ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); >>>> >>>> it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). >>>> >>>> BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. Just create a single KSP in the usual way and use the options >>>> >>>> -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse >>>> >>>> >>>> Barry >>>> >>>> >>>> >>>> >>>> >>>>> >>>>> If there was a way to store the IS containing the ordering ? or any >>>>> other array ? into the shell context (this I know how to do that) and >>>>> the pass the IS directly to the preconditioner instead of using >>>>> PCFactorSetMatOrderingType would be the best solution for me. >>>>> >>>>> Thanks >>>>> >>>>> Gianluca >>>>> >>>>> >>>>> On 12 July 2011 20:15, Barry Smith wrote: >>>>>> >>>>>> On Jul 12, 2011, at 3:47 AM, Gianluca Meneghello wrote: >>>>>> >>>>>>> Jed, thanks for your answer. >>>>>>> >>>>>>> This is exactly what I was looking for (it took me some time to test it...). >>>>>>> >>>>>>> Let me ask you a couple of additional questions: >>>>>>> >>>>>>> 1) in this configuration, is there an alternative way to pass the >>>>>>> ordering to the preconditioner? Becuse my ordering depends on the grid >>>>>>> an not on the solution, it is constant during all the computations and >>>>>>> I have no advantage on calling the function that creates multiple >>>>>>> times (the pc is built and destroyed several times). >>>>>> >>>>>> Not sure exactly what you want here. You can put the ordering information in the shell context and just keep it there forever and don't need to to recompute. >>>>>> >>>>>> PCShellSet/GetContext()? >>>>>>> >>>>>>> 2) what's the best way to make the shell pc selectable from the option >>>>>>> database, so that it can be replaced with more standard ones? >>>>>>> >>>>>> >>>>>> /* here the user can provide -pc_type shell or ilu or whatever */ >>>>>> KSPSetFromOptions(ksp); >>>>>> KSPGetPC(ksp,&pc); >>>>>> /* the following lines will only do something if the user selected the PCType of shell otherwise they are ignored */ >>>>>> >>>>>> PCShellSetContext(pc,....) >>>>>> PCShellSetSetUp(pc,....) >>>>>> PCShellSetApply(pc,.....) >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Gianluca >>>>>>> >>>>>>> On 1 July 2011 14:43, Jed Brown wrote: >>>>>>>> On Fri, Jul 1, 2011 at 07:11, Gianluca Meneghello >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I've tried to implement Jed's solution as it was looking faster to >>>>>>>>> try. This is what I came up with for two different orders, FENOrder >>>>>>>>> and FWNOrder respectively >>>>>>>>> >>>>>>>>> 283 KSP ksp; >>>>>>>>> 284 PC pc; >>>>>>>>> 285 >>>>>>>>> 286 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>>>> 287 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>>>> 288 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>>>> CHKERRQ(ierr); >>>>>>>>> 289 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>>>> 290 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>>>> 291 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>>>> 292 ierr = PCFactorSetMatOrderingType(pc,"FENOrder"); CHKERRQ(ierr); >>>>>>>>> 293 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>>>> 294 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>>>>> 295 >>>>>>>>> 296 ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr); >>>>>>>>> 297 ierr = KSPSetOptionsPrefix(ksp,"mg_"); CHKERRQ(ierr); >>>>>>>>> 298 ierr = KSPSetOperators(ksp,J,J,SAME_NONZERO_PATTERN); >>>>>>>>> CHKERRQ(ierr); >>>>>>>>> 299 ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>>>> 300 ierr = KSPGetPC(ksp,&pc); CHKERRQ(ierr); >>>>>>>>> 301 ierr = PCFactorSetFill(pc,1); CHKERRQ(ierr); >>>>>>>>> 302 ierr = PCFactorSetMatOrderingType(pc,"FWNOrder"); CHKERRQ(ierr); >>>>>>>>> 303 ierr = KSPSolve(ksp,b,x); CHKERRQ(ierr); >>>>>>>>> 304 ierr = KSPDestroy(&ksp); CHKERRQ(ierr); >>>>>>>> >>>>>>>> It looks like you are putting all of this in PCApply_YourShell(). I suggest >>>>>>>> making >>>>>>>> struct YourShellContext { >>>>>>>> PC fenpc,fwnpc; >>>>>>>> Vec work; >>>>>>>> PetscBool issetup; >>>>>>>> }; >>>>>>>> Then pass a pointer to this thing in when you create the PCShell. Then >>>>>>>> something like >>>>>>>> PCSetUp_YourShell(PC pc) { >>>>>>>> struct YourShellContext *ctx; >>>>>>>> Mat A,B; >>>>>>>> MatStructure mstr; >>>>>>>> const char *prefix; >>>>>>>> char iprefix[256]; >>>>>>>> PCShellGetContext(pc,&ctx); >>>>>>>> PCGetOptionsPrefix(pc,&prefix); >>>>>>>> if (!ctx->issetup) { >>>>>>>> ierr = MatGetVecs(A,&ctx->work,PETSC_NULL); >>>>>>>> PCCreate(comm,&ctx->fenpc); >>>>>>>> PCSetType(ctx->fenpc,PCILU); >>>>>>>> PCFactorSetFill(ctx->fenpc,1); >>>>>>>> PCFactorSetMatOrderingType(ctx->fenpc,"FENOrder"); >>>>>>>> snprintf(iprefix,sizeof iprefix,"%sfen",prefix); >>>>>>>> PCSetOptionsPrefix(ctx->fenpc,iprefix); >>>>>>>> PCSetFromOptions(ctx->fenpc); >>>>>>>> /* Same as above for ctx->fwnpc */ >>>>>>>> } >>>>>>>> PCGetOperators(pc,&A,&B,&mstr); >>>>>>>> PCSetOperators(ctx->fenpc,A,B,mstr); >>>>>>>> PCSetOperators(ctx->fwnpc,A,B,mstr); >>>>>>>> PCSetUp(ctx->fenpc); >>>>>>>> PCSetUp(ctx->fwnpc); >>>>>>>> ctx->issetup = PETSC_TRUE; >>>>>>>> } >>>>>>>> Then in PCApply_YourShell(PC pc,Vec X,Vec Y) { >>>>>>>> ... >>>>>>>> PCGetOperators(pc,&A,PETSC_NULL,PETSC_NULL); >>>>>>>> PCApply(ctx->fenpc,X,Y); /* apply first ordering */ >>>>>>>> VecScale(Y,-1.0); >>>>>>>> MatMultAdd(A,Y,X,ctx->work); /* Compute fresh residual */ >>>>>>>> PCApply(ctx->fwnpc,ctx->work,Y); >>>>>>>> } >>>>>>>>> >>>>>>>>> I don't know if this is what you intended... is there a way to make >>>>>>>>> KSP recompute the factorization with the different ordering without >>>>>>>>> destroying and recreating the KSP? >>>>>>>> >>>>>>>> Store both as in the code above. >>>>>>>> >>>>>>>>> >>>>>>>>> Concerning Barry's option, js it possible to use the PETSc ilu >>>>>>>>> factorization function (and eventually the ones provided by external >>>>>>>>> libraries) inside my PCSHELL, passing it the various ordering I will >>>>>>>>> need? If so, what's the best way to access it? >>>>>>>> >>>>>>>> Does the code above answer this question? >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>>>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>>>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>>>>> Amedeo Modigliani >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>>>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>>>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>>>> Amedeo Modigliani >>>> >>>> >>> >>> >>> >>> -- >>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>> Amedeo Modigliani >> From gianmail at gmail.com Mon Jul 18 13:53:18 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Mon, 18 Jul 2011 20:53:18 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing In-Reply-To: <8A766A6C-A0A6-440B-974D-1338FC6424E4@mcs.anl.gov> References: <9F52876E-C796-407A-8608-53D30D666A83@mcs.anl.gov> <86F0EB88-6914-4E20-9775-483E8EC2CBD7@mcs.anl.gov> <715771F6-2966-4703-8F15-E53AB994A16E@mcs.anl.gov> <60D2F23A-E024-4944-82C2-74467374A821@mcs.anl.gov> <48F71658-069D-4C41-8294-E55A53CA6748@gmail.com> <8A766A6C-A0A6-440B-974D-1338FC6424E4@mcs.anl.gov> Message-ID: I will! Thanks for the suggestions Gianluca On Monday, 18 July 2011, Barry Smith wrote: > > ?richardson, you may also need to set that KSP with KSPSetInitialGuessNonzero() > > ? Barry > > > > On Jul 18, 2011, at 1:36 PM, Gianluca Meneghello wrote: > >> Barry, >> I'm actually using ilu as a smoother inside a multigrid cycle. Does in this case make sense to use ksppreonly? Or should I use Richardson? >> >> Thanks >> >> >> >> Il giorno 18/lug/2011, alle ore 20:14, Barry Smith ha scritto: >> >>> >>> On Jul 18, 2011, at 9:43 AM, Gianluca Meneghello wrote: >>> >>>> Barry, >>>> >>>> thanks again for your input. That would be really good. >>>> >>>> When executing my program with the options you suggested I obtain >>>> various errors (unknown pc etc) so I tried to figure out how to avoid >>>> them. Is it something like this what you intended? >>>> >>>> ./main >>>> -mg_pc_type composite >>>> -mg_pc_composite_pcs ilu,ilu >>>> -mg_ksp_type preonly >>> >>> ? I don't think you want this one. It says not to use a Krylov method on the outside; the preconditioner will just apply the two ILU triangular and that is it. ?In other words it won't even solve the linear system. You likely want -mg_ksp_type gmres >>> >>> ?Barry >>> >>>> -mg_sub_0_pc_factor_mat_ordering_type FESOrder >>>> -mg_sub_1_pc_factor_mat_ordering_type FWSOrder >>>> -mg_pc_composite_type multiplicative >>>> -mg_ksp_view >>>> >>>> provided that >>>> >>>> KSPSetOptionsPrefix(ksp,"mg_"); >>>> >>>> has been applied to the KSP I want to modify. >>>> >>>> Thanks again >>>> >>>> Gianluca >>>> >>>> >>>> On 13 July 2011 15:33, Barry Smith wrote: >>>>> >>>>> On Jul 13, 2011, at 4:02 AM, Gianluca Meneghello wrote: >>>>> >>>>>> Thanks Barry. >>>>>> >>>>>> What I'm doing at the moment in order to pass the ordering to the PC is >>>>>> >>>>>> PCFactorSetMatOrderingType(PC,"FESOrder"); >>>>>> >>>>>> where the FESOrder(...) function computes the ordering and has been >>>>>> registered in the main function with >>>>>> >>>>>> MatOrderingRegister("FESOrder",0,"FESOrder",FESOrder); >>>>>> >>>>>> The same is done for al the different orderings I'm using inside the >>>>>> PCSHELL suggested by Jed. >>>>>> >>>>>> My understanding is that the FESOrder(...) function is evaluated every >>>>>> time the PCApply(...) function is called. >>>>> >>>>> Definitely not. Looking at PCSetUp_ILU() >>>>> >>>>> } else { >>>>> ?if (!pc->setupcalled) { >>>>> ? ?/* first time in so compute reordering and symbolic factorization */ >>>>> ? ?ierr = MatGetOrdering(pc->pmat,((PC_Factor*)ilu)->ordering,&ilu->row,&ilu->col);CHKERRQ(ierr); >>>>> >>>>> it will actually call the ordering routine a TOTAL of ONE TIME if you use SAME_NONZERO_PATTERN to KSPSetOperators() or PCSetOperators(). (it is smart enough to reuse the ordering). >>>>> >>>>> BTW: I don't think you even need to use a PCSHELL preconditioner. You can get the same effect with a PCCOMPOSITE. ?Just create a single KSP in the usual way and use the options >>>>> >>>>> -pc_type composite -pc_composite_type ilu,ilu -sub_0_ksp_type preonly -sub_1_ksp_type preonly -sub_0_pc_factor_mat_ordering_type FESOrder -sub_1_pc_factor_mat_ordering_type anotherorder somethingelse >>>>> >>>>> >>>>> Barry >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> If there was a way to store the IS containing the ordering ? or any >>>>>> other array ? into the shell context (this I know how to do that) and >>>>>> the pass the IS directly to the preconditioner instead of using >>>>>> PCFactorSetMatOrderingType would be the best s -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From amesga1 at tigers.lsu.edu Mon Jul 18 16:56:42 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 18 Jul 2011 16:56:42 -0500 Subject: [petsc-users] reading petsc binary vec in python. Message-ID: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> Dear all, I create a petsc binary output using VecView from a structured da grid. How can I read it in python if I know domains size? Best, Ata From ecoon at lanl.gov Mon Jul 18 17:37:43 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Mon, 18 Jul 2011 16:37:43 -0600 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> Message-ID: <1311028663.5755.23.camel@echo.lanl.gov> On Mon, 2011-07-18 at 16:56 -0500, Ataollah Mesgarnejad wrote: > Dear all, > > I create a petsc binary output using VecView from a structured da grid. How can I read it in python if I know domains size? > This is currently a work in progress, and this shouldn't be assumed to be the best way in a few days, but the following works now. It's still a little buggy for matrices, but will get fixed soon. If you're using petsc-dev, you'll want to: In [1]: import sys, os In [2]: sys.path.append(os.path.join(os.environ['PETSC_DIR'],'bin','python')) In [3]: import PetscBinaryRead If you're using a release version of petsc, download the following module to a handy place ($PETSC_DIR/bin/python is where this goes in petsc-dev). http://petsc.cs.iit.edu/petsc/petsc-dev/raw-file/tip/bin/python/PetscBinaryRead.py Add it to your path and import it. Then, a simple: In [4]: petsc_objs = PetscBinaryRead.readBinaryFile('myfile.dat') petsc_objs will now be an N-length list of 2-tuples. N indicates the number of PETSc objects that were "View"ed into the binary file, and each 2-tuple consists of a string to identify what it is ('Vec', 'Mat', or 'IS') and a numpy representation of the data. You then can reshape the array to any shape you want: In [5]: da_vec = petsc_objs[0][1].reshape((NZ, NY, NX, NDOFS)) Note the order! Ethan > Best, > Ata > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From amesga1 at tigers.lsu.edu Mon Jul 18 17:56:05 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 18 Jul 2011 17:56:05 -0500 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <1311028663.5755.23.camel@echo.lanl.gov> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> <1311028663.5755.23.camel@echo.lanl.gov> Message-ID: <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> Thanks Ethan, I did as you said but when I try to read the file using: petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') I get the following error: Traceback (most recent call last): File "./petscvec.py", line 4, in petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') File "/Users/ataollahmesgarnejad/Software/petsc-3.1/bin/python/PetscBinaryRead.py", line 118, in readBinaryFile header = np.fromfile(fid, dtype=IntType, count=1)[0] IndexError: index out of bounds Which is peculiar since when I try to read the header myself using: header=numpy.fromfile('V-018.bin',dtype='>i4',count=1) print header[0] I get: 1211214 which is a Vec identifier. Any thoughts on what goes wrong? Best, Ata From ecoon at lanl.gov Mon Jul 18 18:28:35 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Mon, 18 Jul 2011 17:28:35 -0600 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> <1311028663.5755.23.camel@echo.lanl.gov> <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> Message-ID: <1311031715.5755.35.camel@echo.lanl.gov> Hi Ata, Can you tell me what version of numpy you're using? In some older versions of numpy, a "fromfile" called on a file with no data left to read did not error, but instead returned an empty list. This implementation, since it doesn't know how many objects are in the file, tries to read objects until it gets an error saying the file is empty. Try something for me: change line 119 of PetscBinaryRead.py to read: except MemoryError, IndexError: Does that fix it? Thanks, Ethan On Mon, 2011-07-18 at 17:56 -0500, Ataollah Mesgarnejad wrote: > Thanks Ethan, > > I did as you said but when I try to read the file using: > > petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') > > I get the following error: > > Traceback (most recent call last): > File "./petscvec.py", line 4, in > petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') > File "/Users/ataollahmesgarnejad/Software/petsc-3.1/bin/python/PetscBinaryRead.py", line 118, in readBinaryFile > header = np.fromfile(fid, dtype=IntType, count=1)[0] > IndexError: index out of bounds > > Which is peculiar since when I try to read the header myself using: > > header=numpy.fromfile('V-018.bin',dtype='>i4',count=1) > print header[0] > > I get: > > 1211214 > > which is a Vec identifier. > > Any thoughts on what goes wrong? > > Best, > Ata -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From ckontzialis at lycos.com Mon Jul 18 20:33:35 2011 From: ckontzialis at lycos.com (Kostas Kontzialis) Date: Tue, 19 Jul 2011 04:33:35 +0300 Subject: [petsc-users] Question about ts_ssp Message-ID: <4E24DEEF.9040304@lycos.com> Dear all, I read on the web page the following: TSSSP Explicit strong stability preserving ODE solver Most hyperbolic conservation laws have exact solutions that are total variation diminishing (TVD) or total variation bounded (TVB) although these solutions often contain discontinuities. Spatial discretizations such as Godunov's scheme and high-resolution finite volume methods (TVD limiters, ENO/WENO) are designed to preserve these properties, but they are usually formulated using a forward Euler time discretization or by coupling the space and time discretization as in the classical Lax-Wendroff scheme. When the space and time discretization is coupled, it is very difficult to produce schemes with high temporal accuracy while preserving TVD properties. An alternative is the semidiscrete formulation where we choose a spatial discretization that is TVD with forward Euler and then choose a time discretization that preserves the TVD property. Such integrators are called strong stability preserving (SSP). Let c_eff be the minimum number of function evaluations required to step as far as one step of forward Euler while still being SSP. Some theoretical bounds 1. There are no explicit methods with c_eff > 1. 2. There are no explicit methods beyond order 4 (for nonlinear problems) and c_eff > 0. 3. There are no implicit methods with order greater than 1 and c_eff > 2. This integrator provides Runge-Kutta methods of order 2, 3, and 4 with maximal values of c_eff. More stages allows for larger values of c_eff which improves efficiency. These implementations are low-memory and only use 2 or 3 work vectors regardless of the total number of stages, so e.g. 25-stage 3rd order methods may be an excellent choice. Methods can be chosen with -ts_ssp_type {rks2,rks3,rk104} rks2: Second order methods with any number s>1 of stages. c_eff = (s-1)/s rks3: Third order methods with s=n^2 stages, n>1. c_eff = (s-n)/s rk104: A 10-stage fourth order method. c_eff = 0.6 However, when I write -ts_ssp_type rk53 I get PETSC ERROR: Unknown TS_SSP type rk253 given! Any suggestions? Costas -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Mon Jul 18 20:42:22 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Mon, 18 Jul 2011 20:42:22 -0500 Subject: [petsc-users] Question about ts_ssp In-Reply-To: <4E24DEEF.9040304@lycos.com> References: <4E24DEEF.9040304@lycos.com> Message-ID: > > Methods can be chosen with -ts_ssp_type {rks2,rks3,rk104} > > rks2: Second order methods with any number s>1 of stages. c_eff = (s-1)/s > > rks3: Third order methods with s=n^2 stages, n>1. c_eff = (s-n)/s > > rk104: A 10-stage fourth order method. c_eff = 0.6 > This means that there there are only three options for -ts_ssp_type. Namely, -ts_ssp_type rks2 -ts_ssp_type rks3 -ts_ssp_type rk104 > However, when I write > > -ts_ssp_type rk53 > Therefore, this is not even close to being valid. What are you trying to do? Get 5 work stages? You can see other ssp options with the -h option (probably helps to pipe the output to 'grep'), ./program -ts_type ssp -h | grep ssp -ts_type : TS method (one of) euler beuler cn pseudo gl ssp theta alpha -ts_ssp_type : Type of SSP method (one of) rks2 rks3 rk104 (TSSSPSetType) -ts_ssp_nstages <5>: Number of stages (TSSSPSetNumStages) So, you would probably want the following options, ./program -ts_type ssp -ts_ssp_type rks3 -ts_ssp_nstages 5 but keep in mind that 5 is the default for -ts_ssp_nstages. Hope that helps. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amesga1 at tigers.lsu.edu Mon Jul 18 21:07:29 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 18 Jul 2011 21:07:29 -0500 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <1311031715.5755.35.camel@echo.lanl.gov> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> <1311028663.5755.23.camel@echo.lanl.gov> <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> <1311031715.5755.35.camel@echo.lanl.gov> Message-ID: I'm using 1.5.1 on my Mac with python 2.7. I changed the line and still getting the same error. Let me say this again that it doesn't make any sense since when I do it in python command line np.fromfile works fine and I get the right output. Best, Ata On Jul 18, 2011, at 6:28 PM, Ethan Coon wrote: > Hi Ata, > > Can you tell me what version of numpy you're using? In some older > versions of numpy, a "fromfile" called on a file with no data left to > read did not error, but instead returned an empty list. This > implementation, since it doesn't know how many objects are in the file, > tries to read objects until it gets an error saying the file is empty. > Try something for me: > > change line 119 of PetscBinaryRead.py to read: > > except MemoryError, IndexError: > > Does that fix it? > > Thanks, > > Ethan > > > On Mon, 2011-07-18 at 17:56 -0500, Ataollah Mesgarnejad wrote: >> Thanks Ethan, >> >> I did as you said but when I try to read the file using: >> >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') >> >> I get the following error: >> >> Traceback (most recent call last): >> File "./petscvec.py", line 4, in >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') >> File "/Users/ataollahmesgarnejad/Software/petsc-3.1/bin/python/PetscBinaryRead.py", line 118, in readBinaryFile >> header = np.fromfile(fid, dtype=IntType, count=1)[0] >> IndexError: index out of bounds >> >> Which is peculiar since when I try to read the header myself using: >> >> header=numpy.fromfile('V-018.bin',dtype='>i4',count=1) >> print header[0] >> >> I get: >> >> 1211214 >> >> which is a Vec identifier. >> >> Any thoughts on what goes wrong? >> >> Best, >> Ata > > -- > ------------------------------------ > Ethan Coon > Post-Doctoral Researcher > Applied Mathematics - T-5 > Los Alamos National Laboratory > 505-665-8289 > > http://www.ldeo.columbia.edu/~ecoon/ > ------------------------------------ > From jedbrown at mcs.anl.gov Mon Jul 18 22:21:01 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 18 Jul 2011 20:21:01 -0700 Subject: [petsc-users] Question about ts_ssp In-Reply-To: References: <4E24DEEF.9040304@lycos.com> Message-ID: Note that you cannot do 5 stages with rks3 because 5 is not square (such a method would be in a different class). Use 4 or 9 stages with rks3. On Jul 18, 2011 6:42 PM, "Sean Farley" wrote: >> >> Methods can be chosen with -ts_ssp_type {rks2,rks3,rk104} >> >> rks2: Second order methods with any number s>1 of stages. c_eff = (s-1)/s >> >> rks3: Third order methods with s=n^2 stages, n>1. c_eff = (s-n)/s >> >> rk104: A 10-stage fourth order method. c_eff = 0.6 >> > > This means that there there are only three options for -ts_ssp_type. Namely, > > -ts_ssp_type rks2 > -ts_ssp_type rks3 > -ts_ssp_type rk104 > >> However, when I write >> >> -ts_ssp_type rk53 >> > > Therefore, this is not even close to being valid. What are you trying to do? > Get 5 work stages? You can see other ssp options with the -h option > (probably helps to pipe the output to 'grep'), > > ./program -ts_type ssp -h | grep ssp > -ts_type : TS method (one of) euler beuler cn pseudo gl ssp theta > alpha > -ts_ssp_type : Type of SSP method (one of) rks2 rks3 rk104 > (TSSSPSetType) > -ts_ssp_nstages <5>: Number of stages (TSSSPSetNumStages) > > So, you would probably want the following options, > > ./program -ts_type ssp -ts_ssp_type rks3 -ts_ssp_nstages 5 > > but keep in mind that 5 is the default for -ts_ssp_nstages. > > Hope that helps. > > Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amesga1 at tigers.lsu.edu Tue Jul 19 09:49:59 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Tue, 19 Jul 2011 09:49:59 -0500 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <1311031715.5755.35.camel@echo.lanl.gov> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> <1311028663.5755.23.camel@echo.lanl.gov> <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> <1311031715.5755.35.camel@echo.lanl.gov> Message-ID: <3715D4B9-A2CE-4514-AEEE-0172A6120F7B@tigers.lsu.edu> Dear Ethan, I started deciphering your code for some reason it tries to read header two times and the error comes from the second try! Is it normal for PetscBinaryRead to try to read header twice? Best, Ata On Jul 18, 2011, at 6:28 PM, Ethan Coon wrote: > Hi Ata, > > Can you tell me what version of numpy you're using? In some older > versions of numpy, a "fromfile" called on a file with no data left to > read did not error, but instead returned an empty list. This > implementation, since it doesn't know how many objects are in the file, > tries to read objects until it gets an error saying the file is empty. > Try something for me: > > change line 119 of PetscBinaryRead.py to read: > > except MemoryError, IndexError: > > Does that fix it? > > Thanks, > > Ethan > > > On Mon, 2011-07-18 at 17:56 -0500, Ataollah Mesgarnejad wrote: >> Thanks Ethan, >> >> I did as you said but when I try to read the file using: >> >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') >> >> I get the following error: >> >> Traceback (most recent call last): >> File "./petscvec.py", line 4, in >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') >> File "/Users/ataollahmesgarnejad/Software/petsc-3.1/bin/python/PetscBinaryRead.py", line 118, in readBinaryFile >> header = np.fromfile(fid, dtype=IntType, count=1)[0] >> IndexError: index out of bounds >> >> Which is peculiar since when I try to read the header myself using: >> >> header=numpy.fromfile('V-018.bin',dtype='>i4',count=1) >> print header[0] >> >> I get: >> >> 1211214 >> >> which is a Vec identifier. >> >> Any thoughts on what goes wrong? >> >> Best, >> Ata > > -- > ------------------------------------ > Ethan Coon > Post-Doctoral Researcher > Applied Mathematics - T-5 > Los Alamos National Laboratory > 505-665-8289 > > http://www.ldeo.columbia.edu/~ecoon/ > ------------------------------------ > From ecoon at lanl.gov Tue Jul 19 10:38:43 2011 From: ecoon at lanl.gov (Ethan Coon) Date: Tue, 19 Jul 2011 09:38:43 -0600 Subject: [petsc-users] reading petsc binary vec in python. In-Reply-To: <3715D4B9-A2CE-4514-AEEE-0172A6120F7B@tigers.lsu.edu> References: <305BCC3E-E29E-4C74-9D44-9C6D779A3AD5@tigers.lsu.edu> <1311028663.5755.23.camel@echo.lanl.gov> <421ABF20-7819-4A2E-81F7-64FA94497BBE@tigers.lsu.edu> <1311031715.5755.35.camel@echo.lanl.gov> <3715D4B9-A2CE-4514-AEEE-0172A6120F7B@tigers.lsu.edu> Message-ID: <1311089923.6920.22.camel@echo.lanl.gov> On Tue, 2011-07-19 at 09:49 -0500, Ataollah Mesgarnejad wrote: > Dear Ethan, > > I started deciphering your code for some reason it tries to read header two times and the error comes from the second try! That is the expected behavior. The code cannot know, a priori, how many vecs you have in your file. The first pass reads the first vec, and the second pass tries to read a second PETSc object but gets an empty file. This is expected to error. Line 119, the except line that I asked you to change, is supposed to catch that error and tell the program that the file is empty. However, I screwed up the line in my last email -- it should read: except (MemoryError, IndexError): That should fix your problem, and will be fixed in dev. Ethan > Is it normal for PetscBinaryRead to try to read header twice? > > > Best, > Ata > On Jul 18, 2011, at 6:28 PM, Ethan Coon wrote: > > > Hi Ata, > > > > Can you tell me what version of numpy you're using? In some older > > versions of numpy, a "fromfile" called on a file with no data left to > > read did not error, but instead returned an empty list. This > > implementation, since it doesn't know how many objects are in the file, > > tries to read objects until it gets an error saying the file is empty. > > Try something for me: > > > > change line 119 of PetscBinaryRead.py to read: > > > > except MemoryError, IndexError: > > > > Does that fix it? > > > > Thanks, > > > > Ethan > > > > > > On Mon, 2011-07-18 at 17:56 -0500, Ataollah Mesgarnejad wrote: > >> Thanks Ethan, > >> > >> I did as you said but when I try to read the file using: > >> > >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') > >> > >> I get the following error: > >> > >> Traceback (most recent call last): > >> File "./petscvec.py", line 4, in > >> petsc_objs = PetscBinaryRead.readBinaryFile('V-018.bin') > >> File "/Users/ataollahmesgarnejad/Software/petsc-3.1/bin/python/PetscBinaryRead.py", line 118, in readBinaryFile > >> header = np.fromfile(fid, dtype=IntType, count=1)[0] > >> IndexError: index out of bounds > >> > >> Which is peculiar since when I try to read the header myself using: > >> > >> header=numpy.fromfile('V-018.bin',dtype='>i4',count=1) > >> print header[0] > >> > >> I get: > >> > >> 1211214 > >> > >> which is a Vec identifier. > >> > >> Any thoughts on what goes wrong? > >> > >> Best, > >> Ata > > > > -- > > ------------------------------------ > > Ethan Coon > > Post-Doctoral Researcher > > Applied Mathematics - T-5 > > Los Alamos National Laboratory > > 505-665-8289 > > > > http://www.ldeo.columbia.edu/~ecoon/ > > ------------------------------------ > > > -- ------------------------------------ Ethan Coon Post-Doctoral Researcher Applied Mathematics - T-5 Los Alamos National Laboratory 505-665-8289 http://www.ldeo.columbia.edu/~ecoon/ ------------------------------------ From khalid_eee at yahoo.com Tue Jul 19 23:34:32 2011 From: khalid_eee at yahoo.com (khalid ashraf) Date: Tue, 19 Jul 2011 21:34:32 -0700 (PDT) Subject: [petsc-users] Naturally ordered to DA vector copy Message-ID: <1311136472.39766.YahooMailRC@web112609.mail.gq1.yahoo.com> Hello, In my application, I need to do both Finite element(FEM) and Finite Difference(FD) computations. I am doing the FD using the DA and FEM using natural numbering(x changes fastest, then y then z). My question is what would be the fastest way to copy the FEM vectors to the FD vectors. Looks like DAGetAO may provide a mapping between DA vector layout and natural layout. Since I am not very familiar with programming using AO, just wanted to ask if there is a more efficient/easier way to copy from the naturally ordered vector to the DA vector before delving into AO. If I have to use AO, could you please redirect me to some related example. Thanks in advance. Khalid -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Wed Jul 20 10:42:05 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 20 Jul 2011 12:42:05 -0300 Subject: [petsc-users] Naturally ordered to DA vector copy In-Reply-To: <1311136472.39766.YahooMailRC@web112609.mail.gq1.yahoo.com> References: <1311136472.39766.YahooMailRC@web112609.mail.gq1.yahoo.com> Message-ID: On 20 July 2011 01:34, khalid ashraf wrote: > Hello, > In my application, I need to do both Finite element(FEM) and Finite > Difference(FD) computations. > I am doing the FD using the DA and FEM using natural numbering(x changes > fastest, then y then z). > My question is what would be the fastest way to copy the FEM vectors to the > FD vectors. Looks like > DAGetAO may provide a mapping between DA vector layout and natural layout. > Since I am not very familiar with programming using AO, just wanted to ask > if there is a more efficient/easier way to copy from the naturally ordered > vector to the DA vector before delving into AO. If I have to use AO, could > you please redirect me to some related example. > Thanks in advance. > Khalid DMDAGlobalToNaturalBegin/End and DMDANaturalToGlobalBegin/End -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From adam1.byrd at gmail.com Wed Jul 20 13:58:36 2011 From: adam1.byrd at gmail.com (Adam Byrd) Date: Wed, 20 Jul 2011 14:58:36 -0400 Subject: [petsc-users] Repeated Solves Message-ID: I'm attempting to integrate PETSc into an existing simulation and I have successfully gotten PETSc to handle the first matrix inversion, but I'm having trouble setting it up properly for subsequent inversions. It currently does the inversions without complaining, but gives incorrect output. I believe I need to 'reset' the matrix being inverted. I am inverting the same matrix each time, but the values and nonzero structure change. I create and set up the matrix, ksp, and pc once, then call a routine that fills the matrix and calls KSPSetOperators() before using MatMatSolve again. The routine is called multiple times. I am including the actual code, but here is the outline of what I'm currently trying: //Setup hamiltonian ierr = MatCreate(PETSC_COMM_WORLD, &hamiltonian);CHKERRQ(ierr); ierr = MatSetType(hamiltonian, MATAIJ);CHKERRQ(ierr); ierr = MatSetSizes(hamiltonian, PETSC_DECIDE, PETSC_DECIDE, rows, columns);CHKERRQ(ierr); ierr = MatAssemblyBegin(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); //Setup PETSc Solver ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr); ierr = KSPSetOperators(ksp, hamiltonian, hamiltonian, SAME_NONZERO_PATTERN);CHKERRQ(ierr); ierr = KSPGetPC(ksp, &pc);CHKERRQ(ierr); ierr = PCSetType(pc, PCLU);CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); ierr = test_invert(hamiltonian, inverseHamiltonian, identityMat, pc, ksp, ...);CHKERRQ(ierr); test_invert (looped part): //Several MatSetValue() calls //Setup PETSc Hamiltonian ierr = MatAssemblyBegin(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); //Setup PETSc Solver ierr = KSPSetOperators(ksp, hamiltonian, hamiltonian, DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr); ierr = KSPSetUp(ksp);CHKERRQ(ierr); //Factor and solve ierr = PCFactorGetMatrix(pc, &hamiltonian);CHKERRQ(ierr); ierr = MatMatSolve(hamiltonian, identityMat, inverseHamiltonian);CHKERRQ(ierr); What needs to be done to the matrix to clear it out and use it again? MatZeroEntries doesn't want to work on an unfactored matrix, plus the nonzero structure changes. Do I need to reset the inverseHamiltonian as well? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: petsccntor.zip Type: application/zip Size: 76411 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Jul 20 16:59:49 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 20 Jul 2011 16:59:49 -0500 Subject: [petsc-users] Repeated Solves In-Reply-To: References: Message-ID: <659BAC84-AE4F-48D7-A868-95B75F029055@mcs.anl.gov> 1) Do not call the MatAssembly stuff here, you haven't put values into the matrix so it doesn't need to be assembled > //Setup hamiltonian > ierr = MatCreate(PETSC_COMM_WORLD, &hamiltonian);CHKERRQ(ierr); > ierr = MatSetType(hamiltonian, MATAIJ);CHKERRQ(ierr); > ierr = MatSetSizes(hamiltonian, PETSC_DECIDE, PETSC_DECIDE, rows, columns);CHKERRQ(ierr); > ierr = MatAssemblyBegin(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > ierr = MatAssemblyEnd(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); 2) Do not pull out the factored matrix into the variable hamiltonian in PCFactorGetMatrix(), use some other name for it. > //Setup PETSc Solver > ierr = KSPSetOperators(ksp, hamiltonian, hamiltonian, DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr); > ierr = KSPSetUp(ksp);CHKERRQ(ierr); > > //Factor and solve > ierr = PCFactorGetMatrix(pc, &hamiltonian);CHKERRQ(ierr); > ierr = MatMatSolve(hamiltonian, identityMat, inverseHamiltonian);CHKERRQ(ierr); 3) Call MatZeroEntries() on hamiltonian immediately after you your MatMatSolve() so that it is empty and ready for the next values you are going to put in. Barry On Jul 20, 2011, at 1:58 PM, Adam Byrd wrote: > I'm attempting to integrate PETSc into an existing simulation and I have successfully gotten PETSc to handle the first matrix inversion, but I'm having trouble setting it up properly for subsequent inversions. It currently does the inversions without complaining, but gives incorrect output. I believe I need to 'reset' the matrix being inverted. I am inverting the same matrix each time, but the values and nonzero structure change. I create and set up the matrix, ksp, and pc once, then call a routine that fills the matrix and calls KSPSetOperators() before using MatMatSolve again. The routine is called multiple times. > > I am including the actual code, but here is the outline of what I'm currently trying: > > //Setup hamiltonian > ierr = MatCreate(PETSC_COMM_WORLD, &hamiltonian);CHKERRQ(ierr); > ierr = MatSetType(hamiltonian, MATAIJ);CHKERRQ(ierr); > ierr = MatSetSizes(hamiltonian, PETSC_DECIDE, PETSC_DECIDE, rows, columns);CHKERRQ(ierr); > ierr = MatAssemblyBegin(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > ierr = MatAssemblyEnd(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > > //Setup PETSc Solver > ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr); > ierr = KSPSetOperators(ksp, hamiltonian, hamiltonian, SAME_NONZERO_PATTERN);CHKERRQ(ierr); > ierr = KSPGetPC(ksp, &pc);CHKERRQ(ierr); > ierr = PCSetType(pc, PCLU);CHKERRQ(ierr); > ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); > ierr = test_invert(hamiltonian, inverseHamiltonian, identityMat, pc, ksp, ...);CHKERRQ(ierr); > > > test_invert (looped part): > > //Several MatSetValue() calls > //Setup PETSc Hamiltonian > ierr = MatAssemblyBegin(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > ierr = MatAssemblyEnd(hamiltonian, MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); > > //Setup PETSc Solver > ierr = KSPSetOperators(ksp, hamiltonian, hamiltonian, DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr); > ierr = KSPSetUp(ksp);CHKERRQ(ierr); > > //Factor and solve > ierr = PCFactorGetMatrix(pc, &hamiltonian);CHKERRQ(ierr); > ierr = MatMatSolve(hamiltonian, identityMat, inverseHamiltonian);CHKERRQ(ierr); > > > What needs to be done to the matrix to clear it out and use it again? MatZeroEntries doesn't want to work on an unfactored matrix, plus the nonzero structure changes. Do I need to reset the inverseHamiltonian as well? > > > From clemens.domanig at uibk.ac.at Thu Jul 21 08:53:06 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Thu, 21 Jul 2011 15:53:06 +0200 Subject: [petsc-users] Getting infos out of MUMPS Message-ID: <4E282F42.9040205@uibk.ac.at> Hi, I'm looking for an example how to get informations back from MUMPS. I want the INFO(12) - after factorization (MUMPS manual 4.9.2 page 29). I don't get it how to use this via Petsc. Would MatMumpsSetIcntl be the right choise? Thx - Clemens From hzhang at mcs.anl.gov Thu Jul 21 09:08:16 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 21 Jul 2011 09:08:16 -0500 Subject: [petsc-users] Getting infos out of MUMPS In-Reply-To: <4E282F42.9040205@uibk.ac.at> References: <4E282F42.9040205@uibk.ac.at> Message-ID: Run your code with '-ksp_view' if PETSc KSP is used, e.g., ~petsc/src/ksp/ksp/examples/tutorials/ex2.c: ./ex2 -pc_type lu -pc_factor_mat_solver_package mumps -ksp_view ... MUMPS run parameters: SYM (matrix type): 0 ... INFOG(12) (number of off-diagonal pivots): 0 Hong On Thu, Jul 21, 2011 at 8:53 AM, Clemens Domanig wrote: > Hi, > > I'm looking for an example how to get informations back from MUMPS. I want > the INFO(12) - after factorization (MUMPS manual 4.9.2 page 29). I don't get > it how to use this via Petsc. Would MatMumpsSetIcntl be the right choise? > Thx - Clemens > From gaurish108 at gmail.com Thu Jul 21 12:27:24 2011 From: gaurish108 at gmail.com (Gaurish Telang) Date: Thu, 21 Jul 2011 13:27:24 -0400 Subject: [petsc-users] Trouble with running code on GPU's Message-ID: I have configured PETSc with "--download-f-blas-lapack=1 --download-cuda=1 --download-cusp=1 --download-thrust=1" I am unable to get the tutorial codes on petsc-dev to work on GPU's. For e.g. in src/snes/examples/tutorials ./ex19 -pc_type jacobi -da_vec_type cusp -da_mat_type aijcusp I am getting the error message: " Unknown vector type: cusp! " Below is the complete error message. [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type seehttp://www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html#external! [0]PETSC ERROR: Unknown vector type: cusp! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: unknown HG Date: unknown [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ex19 on a arch-linu named gpute-10 by gaurish.telang Thu Jul 21 13:10:22 2011 [0]PETSC ERROR: Libraries linked from /home/gaurish.telang/petsc-dev/petsc-dev/arch-linux2-c-debug/lib [0]PETSC ERROR: Configure run at Thu Jul 21 12:48:29 2011 [0]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-cuda=1 --download-cusp=1 --download-thrust=1 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecSetType() line 45 in src/vec/vec/interface/vecreg.c [0]PETSC ERROR: DMCreateGlobalVector_DA() line 35 in src/dm/impls/da/dadist.c [0]PETSC ERROR: DMCreateGlobalVector() line 286 in src/dm/interface/dm.c [0]PETSC ERROR: DMMGSetDM() line 258 in src/snes/utils/damg.c [0]PETSC ERROR: main() line 113 in src/snes/examples/tutorials/ex19.c -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 86. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. ------------------------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Jul 21 12:57:15 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 21 Jul 2011 12:57:15 -0500 Subject: [petsc-users] Trouble with running code on GPU's In-Reply-To: References: Message-ID: <5FAEF1CF-E26D-4B47-9957-16129DC49364@mcs.anl.gov> > --download-cuda=1 --download-cusp=1 --download-thrust=1 do not work. You need to install these packages yourself per the instructions and then use the ./configure flags --with-cuda --with-cusp --with-thrust Barry On Jul 21, 2011, at 12:27 PM, Gaurish Telang wrote: > I have configured PETSc with "--download-f-blas-lapack=1 --download-cuda=1 --download-cusp=1 --download-thrust=1" > > I am unable to get the tutorial codes on petsc-dev to work on GPU's. > > For e.g. in src/snes/examples/tutorials > > ./ex19 -pc_type jacobi -da_vec_type cusp -da_mat_type aijcusp > I am getting the error message: " Unknown vector type: cusp! " > > Below is the complete error message. > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type > seehttp:// > www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html#external > ! > [0]PETSC ERROR: Unknown vector type: cusp! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: unknown HG Date: unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./ex19 on a arch-linu named gpute-10 by gaurish.telang Thu Jul 21 13:10:22 2011 > [0]PETSC ERROR: Libraries linked from /home/gaurish.telang/petsc-dev/petsc-dev/arch-linux2-c-debug/lib > [0]PETSC ERROR: Configure run at Thu Jul 21 12:48:29 2011 > [0]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-cuda=1 --download-cusp=1 --download-thrust=1 > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: VecSetType() line 45 in src/vec/vec/interface/vecreg.c > [0]PETSC ERROR: DMCreateGlobalVector_DA() line 35 in src/dm/impls/da/dadist.c > [0]PETSC ERROR: DMCreateGlobalVector() line 286 in src/dm/interface/dm.c > [0]PETSC ERROR: DMMGSetDM() line 258 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 113 in src/snes/examples/tutorials/ex19.c > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 86. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > ------------------------------------------------------------------------ > > > From clemens.domanig at uibk.ac.at Fri Jul 22 01:12:39 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Fri, 22 Jul 2011 08:12:39 +0200 Subject: [petsc-users] Getting infos out of MUMPS In-Reply-To: References: <4E282F42.9040205@uibk.ac.at> Message-ID: <4E2914D7.2080501@uibk.ac.at> So there is no way to get it by calling a nice C-function? Clemens Hong Zhang wrote: > Run your code with '-ksp_view' if PETSc KSP is used, e.g., > ~petsc/src/ksp/ksp/examples/tutorials/ex2.c: > > ./ex2 -pc_type lu -pc_factor_mat_solver_package mumps -ksp_view > ... > MUMPS run parameters: > SYM (matrix type): 0 > ... > INFOG(12) (number of off-diagonal pivots): 0 > Hong > > On Thu, Jul 21, 2011 at 8:53 AM, Clemens Domanig > wrote: >> Hi, >> >> I'm looking for an example how to get informations back from MUMPS. I want >> the INFO(12) - after factorization (MUMPS manual 4.9.2 page 29). I don't get >> it how to use this via Petsc. Would MatMumpsSetIcntl be the right choise? >> Thx - Clemens >> From jedbrown at mcs.anl.gov Fri Jul 22 01:24:54 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 21 Jul 2011 23:24:54 -0700 Subject: [petsc-users] Getting infos out of MUMPS In-Reply-To: <4E2914D7.2080501@uibk.ac.at> References: <4E282F42.9040205@uibk.ac.at> <4E2914D7.2080501@uibk.ac.at> Message-ID: On Thu, Jul 21, 2011 at 23:12, Clemens Domanig wrote: > So there is no way to get it by calling a nice C-function? It is returned by MatGetInertia(F,&nneg,PETSC_NULL,PETSC_NULL); Note that this is INFOG(12), not INFO(12), so it is supposed to be a global number. I don't really know the semantic difference between these, since MUMPS has this insane interface that scatters the whole right hand side to rank 0, and does not synchronize all diagnostics to all ranks, so it's not clear that there is any way to access separate numbers for each process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Fri Jul 22 15:33:00 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Fri, 22 Jul 2011 14:33:00 -0600 Subject: [petsc-users] Conditional Constraints Message-ID: Hi, I am trying to add a constraint feature to my first PETSc application, which uses the finite difference method to calculate the potential distribution produced by a collection of electrodes in a resistive medium. I would like to make this simulation more realistic by imposing a maximum electric current and a maximum potential difference that can be supplied to each electrode by the power supply. If the medium between the electrodes is very conductive, the current maximum would be exceeded by the maximum potential difference, so the potential difference should be decreased from maximum until it produces the maximum current. On the other hand, the potential difference between the electrodes should remain at maximum as long as the current remains below maximum (say, for a less conductive medium). I added an extra degree of freedom (the electrode voltages) to my DMDA, and I developed a set of conditional expressions that describe the above constraints, but one problem is that the logic relies on if-then-else decisions that are made when forming the function/residual and the Jacobian. Once these decisions are made, of course, the conditions are not checked again until the next function or Jacobian evaluation. The non-linear solver then tends to oscillate between extreme solutions to the opposing conditions with each iteration, and never converges towards a reasonable solution. Is there a better strategy for solving such problems? Does PETSc offer mechanisms to aid in their solution? I would very much appreciate any hints. Thank you for your time, Jon From bsmith at mcs.anl.gov Fri Jul 22 15:46:44 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jul 2011 15:46:44 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: <2A5BBF81-4A5E-4E23-AFB2-8E951F0B9F08@mcs.anl.gov> Jon, You may be in luck. In PETSc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have now implemented variational inequality non-linear solvers with box constraints. That is one has a set of variables u and algebraic equations F(u) = 0 (say of size n each) but in addition one has constraints lu <= u <= uu where some or all of lu may be negative infinity (no constraints) and some or all of uu may be infinity (no constraints). There is also a constraint on the sign of F() for those equations associated with active constraints. If your constraints are not in this form sometimes you can introduce new additional variables to get it in this form. Read up a little on variational inequalities on the web. To use this you provide the usual SNES function and Jacobian but you also provide SNESVISetVariableBounds() there is a manual page for this function and for for SNESVI at http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html (the link is broken right now but Satish is fixing it). Plus several examples in src/snes/examples/tutorials (in petsc-dev). This is all new code so we would be interested in your feedback. Barry On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > Hi, > > I am trying to add a constraint feature to my first PETSc application, which uses the finite difference method to calculate the potential distribution produced by a collection of electrodes in a resistive medium. I would like to make this simulation more realistic by imposing a maximum electric current and a maximum potential difference that can be supplied to each electrode by the power supply. If the medium between the electrodes is very conductive, the current maximum would be exceeded by the maximum potential difference, so the potential difference should be decreased from maximum until it produces the maximum current. On the other hand, the potential difference between the electrodes should remain at maximum as long as the current remains below maximum (say, for a less conductive medium). > > I added an extra degree of freedom (the electrode voltages) to my DMDA, and I developed a set of conditional expressions that describe the above constraints, but one problem is that the logic relies on if-then-else decisions that are made when forming the function/residual and the Jacobian. Once these decisions are made, of course, the conditions are not checked again until the next function or Jacobian evaluation. The non-linear solver then tends to oscillate between extreme solutions to the opposing conditions with each iteration, and never converges towards a reasonable solution. > > Is there a better strategy for solving such problems? Does PETSc offer mechanisms to aid in their solution? I would very much appreciate any hints. > > Thank you for your time, > > Jon From jbacks at mcmillan-mcgee.com Fri Jul 22 16:16:23 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Fri, 22 Jul 2011 15:16:23 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <2A5BBF81-4A5E-4E23-AFB2-8E951F0B9F08@mcs.anl.gov> References: <2A5BBF81-4A5E-4E23-AFB2-8E951F0B9F08@mcs.anl.gov> Message-ID: <0FE147DE-3B09-4776-9D03-F82C5606880F@mcmillan-mcgee.com> Barry, Thank you so much for your response. Lucky, indeed! I look forward to trying out these new features. I neglected to mention in my original post that my electrical problem is part of a DAE, which includes a time-dependent heating problem. Can SNESVI constraints be used in conjunction with TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only need the constraints for the time-independent electrical portion.) Thank you again, Jon On 2011-07-22, at 2:46 PM, Barry Smith wrote: > > Jon, > > You may be in luck. In PETSc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have now implemented variational inequality non-linear solvers with box constraints. > > That is one has a set of variables u and algebraic equations F(u) = 0 (say of size n each) but in addition one has constraints lu <= u <= uu where some or all of lu may be negative infinity (no constraints) and some or all of uu may be infinity (no constraints). There is also a constraint on the sign of F() for those equations associated with active constraints. If your constraints are not in this form sometimes you can introduce new additional variables to get it in this form. Read up a little on variational inequalities on the web. > > To use this you provide the usual SNES function and Jacobian but you also provide SNESVISetVariableBounds() there is a manual page for this function and for for SNESVI at http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html (the link is broken right now but Satish is fixing it). Plus several examples in src/snes/examples/tutorials (in petsc-dev). > > This is all new code so we would be interested in your feedback. > > > Barry > > > > > > On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > >> Hi, >> >> I am trying to add a constraint feature to my first PETSc application, which uses the finite difference method to calculate the potential distribution produced by a collection of electrodes in a resistive medium. I would like to make this simulation more realistic by imposing a maximum electric current and a maximum potential difference that can be supplied to each electrode by the power supply. If the medium between the electrodes is very conductive, the current maximum would be exceeded by the maximum potential difference, so the potential difference should be decreased from maximum until it produces the maximum current. On the other hand, the potential difference between the electrodes should remain at maximum as long as the current remains below maximum (say, for a less conductive medium). >> >> I added an extra degree of freedom (the electrode voltages) to my DMDA, and I developed a set of conditional expressions that describe the above constraints, but one problem is that the logic relies on if-then-else decisions that are made when forming the function/residual and the Jacobian. Once these decisions are made, of course, the conditions are not checked again until the next function or Jacobian evaluation. The non-linear solver then tends to oscillate between extreme solutions to the opposing conditions with each iteration, and never converges towards a reasonable solution. >> >> Is there a better strategy for solving such problems? Does PETSc offer mechanisms to aid in their solution? I would very much appreciate any hints. >> >> Thank you for your time, >> >> Jon > From bsmith at mcs.anl.gov Fri Jul 22 16:20:04 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jul 2011 16:20:04 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: <0FE147DE-3B09-4776-9D03-F82C5606880F@mcmillan-mcgee.com> References: <2A5BBF81-4A5E-4E23-AFB2-8E951F0B9F08@mcs.anl.gov> <0FE147DE-3B09-4776-9D03-F82C5606880F@mcmillan-mcgee.com> Message-ID: <256F292E-BD6A-48D5-A0BF-37382E9C15F3@mcs.anl.gov> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > Barry, > > Thank you so much for your response. Lucky, indeed! I look forward to trying out these new features. > > I neglected to mention in my original post that my electrical problem is part of a DAE, which includes a time-dependent heating problem. Can SNESVI constraints be used in conjunction with TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only need the constraints for the time-independent electrical portion.) We have not yet put that in but Shri is starting to look at that just now. Basically we would have a TSVISetVariableBounds() and handle everything from there. I suggest you start with a simple time electrical portion with constraints to explore and we'll go from there. Shri is actually a electrical networks guy and so can speak your language. Barry > > Thank you again, > > Jon > > On 2011-07-22, at 2:46 PM, Barry Smith wrote: > >> >> Jon, >> >> You may be in luck. In PETSc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have now implemented variational inequality non-linear solvers with box constraints. >> >> That is one has a set of variables u and algebraic equations F(u) = 0 (say of size n each) but in addition one has constraints lu <= u <= uu where some or all of lu may be negative infinity (no constraints) and some or all of uu may be infinity (no constraints). There is also a constraint on the sign of F() for those equations associated with active constraints. If your constraints are not in this form sometimes you can introduce new additional variables to get it in this form. Read up a little on variational inequalities on the web. >> >> To use this you provide the usual SNES function and Jacobian but you also provide SNESVISetVariableBounds() there is a manual page for this function and for for SNESVI at http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html (the link is broken right now but Satish is fixing it). Plus several examples in src/snes/examples/tutorials (in petsc-dev). >> >> This is all new code so we would be interested in your feedback. >> >> >> Barry >> >> >> >> >> >> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >> >>> Hi, >>> >>> I am trying to add a constraint feature to my first PETSc application, which uses the finite difference method to calculate the potential distribution produced by a collection of electrodes in a resistive medium. I would like to make this simulation more realistic by imposing a maximum electric current and a maximum potential difference that can be supplied to each electrode by the power supply. If the medium between the electrodes is very conductive, the current maximum would be exceeded by the maximum potential difference, so the potential difference should be decreased from maximum until it produces the maximum current. On the other hand, the potential difference between the electrodes should remain at maximum as long as the current remains below maximum (say, for a less conductive medium). >>> >>> I added an extra degree of freedom (the electrode voltages) to my DMDA, and I developed a set of conditional expressions that describe the above constraints, but one problem is that the logic relies on if-then-else decisions that are made when forming the function/residual and the Jacobian. Once these decisions are made, of course, the conditions are not checked again until the next function or Jacobian evaluation. The non-linear solver then tends to oscillate between extreme solutions to the opposing conditions with each iteration, and never converges towards a reasonable solution. >>> >>> Is there a better strategy for solving such problems? Does PETSc offer mechanisms to aid in their solution? I would very much appreciate any hints. >>> >>> Thank you for your time, >>> >>> Jon >> > From kylewendt at gmail.com Sat Jul 23 13:22:54 2011 From: kylewendt at gmail.com (Kyle Wendt) Date: Sat, 23 Jul 2011 14:22:54 -0400 Subject: [petsc-users] MatMatMul with mpidense Message-ID: I am trying to multiple two mpidense matrices with matmatmil and I get the following runtime error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. I have built and linked petsc again scalapack and blacs but this have not helped, is this operation really not supported? From balay at mcs.anl.gov Sat Jul 23 13:44:18 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 23 Jul 2011 13:44:18 -0500 (CDT) Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: Message-ID: you need --download-plapack for this. [petsc does not use scalapack. Its a dependency of mumps - so you need it only if using --download-mumps] Satish On Sat, 23 Jul 2011, Kyle Wendt wrote: > I am trying to multiple two mpidense matrices with matmatmil and I get > the following runtime error: > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > 13:37:48 CDT 2011 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > I have built and linked petsc again scalapack and blacs but this have > not helped, is this operation really not supported? > From kylewendt at gmail.com Sat Jul 23 14:12:46 2011 From: kylewendt at gmail.com (Kyle Wendt) Date: Sat, 23 Jul 2011 15:12:46 -0400 Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: Message-ID: I also tried that and when i run my code i get this error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Error in external library! [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently supported! On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay wrote: > you need --download-plapack for this. > > [petsc does not use scalapack. Its a dependency of mumps - so you need > it only if using --download-mumps] > > Satish > > On Sat, 23 Jul 2011, Kyle Wendt wrote: > >> I am trying to multiple two mpidense matrices with matmatmil and I get >> the following runtime error: >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: No support for this operation for this object type! >> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 >> 13:37:48 CDT 2011 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> >> I have built and linked petsc again scalapack and blacs but this have >> not helped, is this operation really not supported? >> > > From bsmith at mcs.anl.gov Sat Jul 23 15:09:33 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 23 Jul 2011 15:09:33 -0500 Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: Message-ID: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> On Jul 23, 2011, at 2:12 PM, Kyle Wendt wrote: > I also tried that and when i run my code i get this error: > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Error in external library! > [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently supported! The error message says it all. plapack is buggy and not supported. Currently there no good options for general purpose parallel dense matrix operations in PETSc. Sorry, our focus is on sparse matrix computations because we can't do everything so we do what we do best. Barry > > On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay wrote: >> you need --download-plapack for this. >> >> [petsc does not use scalapack. Its a dependency of mumps - so you need >> it only if using --download-mumps] >> >> Satish >> >> On Sat, 23 Jul 2011, Kyle Wendt wrote: >> >>> I am trying to multiple two mpidense matrices with matmatmil and I get >>> the following runtime error: >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: No support for this operation for this object type! >>> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 >>> 13:37:48 CDT 2011 >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> >>> I have built and linked petsc again scalapack and blacs but this have >>> not helped, is this operation really not supported? >>> >> >> From kylewendt at gmail.com Sat Jul 23 15:21:17 2011 From: kylewendt at gmail.com (Kyle Wendt) Date: Sat, 23 Jul 2011 16:21:17 -0400 Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> References: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> Message-ID: Ok, is there a way to get more detailed information as to what went wrong? On Sat, Jul 23, 2011 at 4:09 PM, Barry Smith wrote: > > On Jul 23, 2011, at 2:12 PM, Kyle Wendt wrote: > >> I also tried that and when i run my code i get this error: >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Error in external library! >> [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently supported! > > ? The error message says it all. plapack is buggy and not supported. Currently there no good options for general purpose parallel dense matrix operations in PETSc. Sorry, our focus is on sparse matrix computations because we can't do everything so we do what we do best. > > ? Barry > >> >> On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay wrote: >>> you need --download-plapack for this. >>> >>> [petsc does not use scalapack. Its a dependency of mumps - so you need >>> it only if using --download-mumps] >>> >>> Satish >>> >>> On Sat, 23 Jul 2011, Kyle Wendt wrote: >>> >>>> I am trying to multiple two mpidense matrices with matmatmil and I get >>>> the following runtime error: >>>> [0]PETSC ERROR: --------------------- Error Message >>>> ------------------------------------ >>>> [0]PETSC ERROR: No support for this operation for this object type! >>>> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 >>>> 13:37:48 CDT 2011 >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>> >>>> I have built and linked petsc again scalapack and blacs but this have >>>> not helped, is this operation really not supported? >>>> >>> >>> > > From balay at mcs.anl.gov Sat Jul 23 16:18:15 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 23 Jul 2011 16:18:15 -0500 (CDT) Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> Message-ID: the error is from petsc code saying this codepath to plapack is buggy. If you wish to debug plapack - you can comment out the following line in src/mat/impls/dense/mpi/mpidense.c SETERRQ(PETSC_ERR_LIB,"Due to apparent bugs in PLAPACK,this is not currently supported"); and then figureout if/why you are getting wrong numbers out of this call.. Satish On Sat, 23 Jul 2011, Kyle Wendt wrote: > Ok, is there a way to get more detailed information as to what went wrong? > > On Sat, Jul 23, 2011 at 4:09 PM, Barry Smith wrote: > > > > On Jul 23, 2011, at 2:12 PM, Kyle Wendt wrote: > > > >> I also tried that and when i run my code i get this error: > >> [0]PETSC ERROR: --------------------- Error Message > >> ------------------------------------ > >> [0]PETSC ERROR: Error in external library! > >> [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently supported! > > > > ? The error message says it all. plapack is buggy and not supported. Currently there no good options for general purpose parallel dense matrix operations in PETSc. Sorry, our focus is on sparse matrix computations because we can't do everything so we do what we do best. > > > > ? Barry > > > >> > >> On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay wrote: > >>> you need --download-plapack for this. > >>> > >>> [petsc does not use scalapack. Its a dependency of mumps - so you need > >>> it only if using --download-mumps] > >>> > >>> Satish > >>> > >>> On Sat, 23 Jul 2011, Kyle Wendt wrote: > >>> > >>>> I am trying to multiple two mpidense matrices with matmatmil and I get > >>>> the following runtime error: > >>>> [0]PETSC ERROR: --------------------- Error Message > >>>> ------------------------------------ > >>>> [0]PETSC ERROR: No support for this operation for this object type! > >>>> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! > >>>> [0]PETSC ERROR: > >>>> ------------------------------------------------------------------------ > >>>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > >>>> 13:37:48 CDT 2011 > >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. > >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > >>>> > >>>> I have built and linked petsc again scalapack and blacs but this have > >>>> not helped, is this operation really not supported? > >>>> > >>> > >>> > > > > > From kylewendt at gmail.com Sat Jul 23 16:59:45 2011 From: kylewendt at gmail.com (Kyle Wendt) Date: Sat, 23 Jul 2011 17:59:45 -0400 Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> Message-ID: Thanks, I'll look into what going wrong over the next few days. On Sat, Jul 23, 2011 at 5:18 PM, Satish Balay wrote: > the error is from petsc code saying this codepath to plapack is buggy. > > If you wish to debug plapack - you can comment out the following line > in src/mat/impls/dense/mpi/mpidense.c > > ?SETERRQ(PETSC_ERR_LIB,"Due to apparent bugs in PLAPACK,this is not currently supported"); > > and then figureout if/why you are getting wrong numbers out of this call.. > > Satish > > On Sat, 23 Jul 2011, Kyle Wendt wrote: > >> Ok, is there a way to get more detailed information as to what went wrong? >> >> On Sat, Jul 23, 2011 at 4:09 PM, Barry Smith wrote: >> > >> > On Jul 23, 2011, at 2:12 PM, Kyle Wendt wrote: >> > >> >> I also tried that and when i run my code i get this error: >> >> [0]PETSC ERROR: --------------------- Error Message >> >> ------------------------------------ >> >> [0]PETSC ERROR: Error in external library! >> >> [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently supported! >> > >> > ? The error message says it all. plapack is buggy and not supported. Currently there no good options for general purpose parallel dense matrix operations in PETSc. Sorry, our focus is on sparse matrix computations because we can't do everything so we do what we do best. >> > >> > ? Barry >> > >> >> >> >> On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay wrote: >> >>> you need --download-plapack for this. >> >>> >> >>> [petsc does not use scalapack. Its a dependency of mumps - so you need >> >>> it only if using --download-mumps] >> >>> >> >>> Satish >> >>> >> >>> On Sat, 23 Jul 2011, Kyle Wendt wrote: >> >>> >> >>>> I am trying to multiple two mpidense matrices with matmatmil and I get >> >>>> the following runtime error: >> >>>> [0]PETSC ERROR: --------------------- Error Message >> >>>> ------------------------------------ >> >>>> [0]PETSC ERROR: No support for this operation for this object type! >> >>>> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! >> >>>> [0]PETSC ERROR: >> >>>> ------------------------------------------------------------------------ >> >>>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 >> >>>> 13:37:48 CDT 2011 >> >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> >>>> >> >>>> I have built and linked petsc again scalapack and blacs but this have >> >>>> not helped, is this operation really not supported? >> >>>> >> >>> >> >>> >> > >> > >> > From knepley at gmail.com Sat Jul 23 18:45:19 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 23 Jul 2011 23:45:19 +0000 Subject: [petsc-users] MatMatMul with mpidense In-Reply-To: References: <4C4B6137-08E1-44B1-91DD-9C58F4D77D81@mcs.anl.gov> Message-ID: On Sat, Jul 23, 2011 at 9:59 PM, Kyle Wendt wrote: > Thanks, I'll look into what going wrong over the next few days. I wrote the initial PLAPACK implementation, and it is very simple. If you are truly going to spend time on the code, your time is better spent replacing PLAPACK with its successor, Elemental http://code.google.com/p/elemental/ than with debugging old PLAPACK code. It will take the same amount of time (or less), perform better, and provide every other user a benefit. Thanks, Matt > On Sat, Jul 23, 2011 at 5:18 PM, Satish Balay wrote: > > the error is from petsc code saying this codepath to plapack is buggy. > > > > If you wish to debug plapack - you can comment out the following line > > in src/mat/impls/dense/mpi/mpidense.c > > > > SETERRQ(PETSC_ERR_LIB,"Due to apparent bugs in PLAPACK,this is not > currently supported"); > > > > and then figureout if/why you are getting wrong numbers out of this > call.. > > > > Satish > > > > On Sat, 23 Jul 2011, Kyle Wendt wrote: > > > >> Ok, is there a way to get more detailed information as to what went > wrong? > >> > >> On Sat, Jul 23, 2011 at 4:09 PM, Barry Smith > wrote: > >> > > >> > On Jul 23, 2011, at 2:12 PM, Kyle Wendt wrote: > >> > > >> >> I also tried that and when i run my code i get this error: > >> >> [0]PETSC ERROR: --------------------- Error Message > >> >> ------------------------------------ > >> >> [0]PETSC ERROR: Error in external library! > >> >> [0]PETSC ERROR: Due to apparent bugs in PLAPACK,this is not currently > supported! > >> > > >> > The error message says it all. plapack is buggy and not supported. > Currently there no good options for general purpose parallel dense matrix > operations in PETSc. Sorry, our focus is on sparse matrix computations > because we can't do everything so we do what we do best. > >> > > >> > Barry > >> > > >> >> > >> >> On Sat, Jul 23, 2011 at 2:44 PM, Satish Balay > wrote: > >> >>> you need --download-plapack for this. > >> >>> > >> >>> [petsc does not use scalapack. Its a dependency of mumps - so you > need > >> >>> it only if using --download-mumps] > >> >>> > >> >>> Satish > >> >>> > >> >>> On Sat, 23 Jul 2011, Kyle Wendt wrote: > >> >>> > >> >>>> I am trying to multiple two mpidense matrices with matmatmil and I > get > >> >>>> the following runtime error: > >> >>>> [0]PETSC ERROR: --------------------- Error Message > >> >>>> ------------------------------------ > >> >>>> [0]PETSC ERROR: No support for this operation for this object type! > >> >>>> [0]PETSC ERROR: MatMatMult not supported for B of type mpidense! > >> >>>> [0]PETSC ERROR: > >> >>>> > ------------------------------------------------------------------------ > >> >>>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > >> >>>> 13:37:48 CDT 2011 > >> >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. > >> >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > >> >>>> > >> >>>> I have built and linked petsc again scalapack and blacs but this > have > >> >>>> not helped, is this operation really not supported? > >> >>>> > >> >>> > >> >>> > >> > > >> > > >> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Mon Jul 25 12:58:11 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Mon, 25 Jul 2011 12:58:11 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: <256F292E-BD6A-48D5-A0BF-37382E9C15F3@mcs.anl.gov> Message-ID: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> ----- Original Message ----- > On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > > > Barry, > > > > Thank you so much for your response. Lucky, indeed! I look forward > > to trying out these new features. > > > > I neglected to mention in my original post that my electrical > > problem is part of a DAE, which includes a time-dependent heating > > problem. Can SNESVI constraints be used in conjunction with > > TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only > > need the constraints for the time-independent electrical portion.) > > We have not yet put that in but Shri is starting to look at that just > now. Basically we would have a TSVISetVariableBounds() and handle > everything from there. I suggest you start with a simple time > electrical portion with constraints to explore and we'll go from > there. Shri is actually a electrical networks guy and so can speak > your language. I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space (-snes_vi_type rs). iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. By the way, does your problem have bounds on the variables or bounds on some function of the variables? Shri > > Barry > > > > > > > Thank you again, > > > > Jon > > > > On 2011-07-22, at 2:46 PM, Barry Smith wrote: > > > >> > >> Jon, > >> > >> You may be in luck. In PETSc-dev > >> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we > >> have now implemented variational inequality non-linear solvers > >> with box constraints. > >> > >> That is one has a set of variables u and algebraic equations > >> F(u) = 0 (say of size n each) but in addition one has > >> constraints lu <= u <= uu where some or all of lu may be > >> negative infinity (no constraints) and some or all of uu may be > >> infinity (no constraints). There is also a constraint on the > >> sign of F() for those equations associated with active > >> constraints. If your constraints are not in this form sometimes > >> you can introduce new additional variables to get it in this > >> form. Read up a little on variational inequalities on the web. > >> > >> To use this you provide the usual SNES function and Jacobian but > >> you also provide SNESVISetVariableBounds() there is a manual > >> page for this function and for for SNESVI at > >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html > >> (the link is broken right now but Satish is fixing it). Plus > >> several examples in src/snes/examples/tutorials (in petsc-dev). > >> > >> This is all new code so we would be interested in your feedback. > >> > >> > >> Barry > >> > >> > >> > >> > >> > >> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > >> > >>> Hi, > >>> > >>> I am trying to add a constraint feature to my first PETSc > >>> application, which uses the finite difference method to calculate > >>> the potential distribution produced by a collection of electrodes > >>> in a resistive medium. I would like to make this simulation more > >>> realistic by imposing a maximum electric current and a maximum > >>> potential difference that can be supplied to each electrode by the > >>> power supply. If the medium between the electrodes is very > >>> conductive, the current maximum would be exceeded by the maximum > >>> potential difference, so the potential difference should be > >>> decreased from maximum until it produces the maximum current. On > >>> the other hand, the potential difference between the electrodes > >>> should remain at maximum as long as the current remains below > >>> maximum (say, for a less conductive medium). > >>> > >>> I added an extra degree of freedom (the electrode voltages) to my > >>> DMDA, and I developed a set of conditional expressions that > >>> describe the above constraints, but one problem is that the logic > >>> relies on if-then-else decisions that are made when forming the > >>> function/residual and the Jacobian. Once these decisions are made, > >>> of course, the conditions are not checked again until the next > >>> function or Jacobian evaluation. The non-linear solver then tends > >>> to oscillate between extreme solutions to the opposing conditions > >>> with each iteration, and never converges towards a reasonable > >>> solution. > >>> > >>> Is there a better strategy for solving such problems? Does PETSc > >>> offer mechanisms to aid in their solution? I would very much > >>> appreciate any hints. > >>> > >>> Thank you for your time, > >>> > >>> Jon > >> > > From jbacks at mcmillan-mcgee.com Mon Jul 25 17:50:32 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Mon, 25 Jul 2011 16:50:32 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> References: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> Message-ID: <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> Hi Shri, Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? Thank you again, Jon On 2011-07-25, at 11:58 AM, Shri wrote: > > ----- Original Message ----- >> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >> >>> Barry, >>> >>> Thank you so much for your response. Lucky, indeed! I look forward >>> to trying out these new features. >>> >>> I neglected to mention in my original post that my electrical >>> problem is part of a DAE, which includes a time-dependent heating >>> problem. Can SNESVI constraints be used in conjunction with >>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>> need the constraints for the time-independent electrical portion.) >> >> We have not yet put that in but Shri is starting to look at that just >> now. Basically we would have a TSVISetVariableBounds() and handle >> everything from there. I suggest you start with a simple time >> electrical portion with constraints to explore and we'll go from >> there. Shri is actually a electrical networks guy and so can speak >> your language. > > > I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. > A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). > i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). > ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space > (-snes_vi_type rs). > iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. > > By the way, does your problem have bounds on the variables or bounds on some function of the variables? > > Shri > > > >> >> Barry >> >> >> >>> >>> Thank you again, >>> >>> Jon >>> >>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>> >>>> >>>> Jon, >>>> >>>> You may be in luck. In PETSc-dev >>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>> have now implemented variational inequality non-linear solvers >>>> with box constraints. >>>> >>>> That is one has a set of variables u and algebraic equations >>>> F(u) = 0 (say of size n each) but in addition one has >>>> constraints lu <= u <= uu where some or all of lu may be >>>> negative infinity (no constraints) and some or all of uu may be >>>> infinity (no constraints). There is also a constraint on the >>>> sign of F() for those equations associated with active >>>> constraints. If your constraints are not in this form sometimes >>>> you can introduce new additional variables to get it in this >>>> form. Read up a little on variational inequalities on the web. >>>> >>>> To use this you provide the usual SNES function and Jacobian but >>>> you also provide SNESVISetVariableBounds() there is a manual >>>> page for this function and for for SNESVI at >>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>> (the link is broken right now but Satish is fixing it). Plus >>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>> >>>> This is all new code so we would be interested in your feedback. >>>> >>>> >>>> Barry >>>> >>>> >>>> >>>> >>>> >>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>> >>>>> Hi, >>>>> >>>>> I am trying to add a constraint feature to my first PETSc >>>>> application, which uses the finite difference method to calculate >>>>> the potential distribution produced by a collection of electrodes >>>>> in a resistive medium. I would like to make this simulation more >>>>> realistic by imposing a maximum electric current and a maximum >>>>> potential difference that can be supplied to each electrode by the >>>>> power supply. If the medium between the electrodes is very >>>>> conductive, the current maximum would be exceeded by the maximum >>>>> potential difference, so the potential difference should be >>>>> decreased from maximum until it produces the maximum current. On >>>>> the other hand, the potential difference between the electrodes >>>>> should remain at maximum as long as the current remains below >>>>> maximum (say, for a less conductive medium). >>>>> >>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>> DMDA, and I developed a set of conditional expressions that >>>>> describe the above constraints, but one problem is that the logic >>>>> relies on if-then-else decisions that are made when forming the >>>>> function/residual and the Jacobian. Once these decisions are made, >>>>> of course, the conditions are not checked again until the next >>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>> to oscillate between extreme solutions to the opposing conditions >>>>> with each iteration, and never converges towards a reasonable >>>>> solution. >>>>> >>>>> Is there a better strategy for solving such problems? Does PETSc >>>>> offer mechanisms to aid in their solution? I would very much >>>>> appreciate any hints. >>>>> >>>>> Thank you for your time, >>>>> >>>>> Jon >>>> >>> > From bsmith at mcs.anl.gov Mon Jul 25 21:58:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 25 Jul 2011 21:58:00 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> References: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> Message-ID: On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > Hi Shri, > > Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. > > For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? > You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. Barry > Thank you again, > > Jon > > On 2011-07-25, at 11:58 AM, Shri wrote: > >> >> ----- Original Message ----- >>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>> >>>> Barry, >>>> >>>> Thank you so much for your response. Lucky, indeed! I look forward >>>> to trying out these new features. >>>> >>>> I neglected to mention in my original post that my electrical >>>> problem is part of a DAE, which includes a time-dependent heating >>>> problem. Can SNESVI constraints be used in conjunction with >>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>>> need the constraints for the time-independent electrical portion.) >>> >>> We have not yet put that in but Shri is starting to look at that just >>> now. Basically we would have a TSVISetVariableBounds() and handle >>> everything from there. I suggest you start with a simple time >>> electrical portion with constraints to explore and we'll go from >>> there. Shri is actually a electrical networks guy and so can speak >>> your language. >> >> >> I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. >> A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). >> i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). >> ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space >> (-snes_vi_type rs). >> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. >> >> By the way, does your problem have bounds on the variables or bounds on some function of the variables? >> >> Shri >> >> >> >>> >>> Barry >>> >>> >>> >>>> >>>> Thank you again, >>>> >>>> Jon >>>> >>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>> >>>>> >>>>> Jon, >>>>> >>>>> You may be in luck. In PETSc-dev >>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>> have now implemented variational inequality non-linear solvers >>>>> with box constraints. >>>>> >>>>> That is one has a set of variables u and algebraic equations >>>>> F(u) = 0 (say of size n each) but in addition one has >>>>> constraints lu <= u <= uu where some or all of lu may be >>>>> negative infinity (no constraints) and some or all of uu may be >>>>> infinity (no constraints). There is also a constraint on the >>>>> sign of F() for those equations associated with active >>>>> constraints. If your constraints are not in this form sometimes >>>>> you can introduce new additional variables to get it in this >>>>> form. Read up a little on variational inequalities on the web. >>>>> >>>>> To use this you provide the usual SNES function and Jacobian but >>>>> you also provide SNESVISetVariableBounds() there is a manual >>>>> page for this function and for for SNESVI at >>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>> (the link is broken right now but Satish is fixing it). Plus >>>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>>> >>>>> This is all new code so we would be interested in your feedback. >>>>> >>>>> >>>>> Barry >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am trying to add a constraint feature to my first PETSc >>>>>> application, which uses the finite difference method to calculate >>>>>> the potential distribution produced by a collection of electrodes >>>>>> in a resistive medium. I would like to make this simulation more >>>>>> realistic by imposing a maximum electric current and a maximum >>>>>> potential difference that can be supplied to each electrode by the >>>>>> power supply. If the medium between the electrodes is very >>>>>> conductive, the current maximum would be exceeded by the maximum >>>>>> potential difference, so the potential difference should be >>>>>> decreased from maximum until it produces the maximum current. On >>>>>> the other hand, the potential difference between the electrodes >>>>>> should remain at maximum as long as the current remains below >>>>>> maximum (say, for a less conductive medium). >>>>>> >>>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>>> DMDA, and I developed a set of conditional expressions that >>>>>> describe the above constraints, but one problem is that the logic >>>>>> relies on if-then-else decisions that are made when forming the >>>>>> function/residual and the Jacobian. Once these decisions are made, >>>>>> of course, the conditions are not checked again until the next >>>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>>> to oscillate between extreme solutions to the opposing conditions >>>>>> with each iteration, and never converges towards a reasonable >>>>>> solution. >>>>>> >>>>>> Is there a better strategy for solving such problems? Does PETSc >>>>>> offer mechanisms to aid in their solution? I would very much >>>>>> appreciate any hints. >>>>>> >>>>>> Thank you for your time, >>>>>> >>>>>> Jon >>>>> >>>> >> > From vijay.m at gmail.com Mon Jul 25 22:17:37 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Mon, 25 Jul 2011 22:17:37 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> Message-ID: Barry/Shri, Is the new VI SNES type in general terms a constrained nonlinear solver ? In other words, given a certain constraint as a function of the variables of interest, does it find the optimal solution that both minimizes the residual and satisfy the constraint ? I have been looking at L-BFGS constrained optimization methods and from what I understand, the variational inequality solver seems to fall under a similar but much broader category. I would much appreciate it if you can point me to some papers related to the method. And if I misunderstood the domain of applicability, please do feel free to correct me. Thanks, Vijay On Mon, Jul 25, 2011 at 9:58 PM, Barry Smith wrote: > > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > >> Hi Shri, >> >> Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. >> >> For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? >> > > ? You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. > > ? Barry > >> Thank you again, >> >> Jon >> >> On 2011-07-25, at 11:58 AM, Shri wrote: >> >>> >>> ----- Original Message ----- >>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>>> >>>>> Barry, >>>>> >>>>> Thank you so much for your response. Lucky, indeed! I look forward >>>>> to trying out these new features. >>>>> >>>>> I neglected to mention in my original post that my electrical >>>>> problem is part of a DAE, which includes a time-dependent heating >>>>> problem. Can SNESVI constraints be used in conjunction with >>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>>>> need the constraints for the time-independent electrical portion.) >>>> >>>> We have not yet put that in but Shri is starting to look at that just >>>> now. Basically we would have a TSVISetVariableBounds() and handle >>>> everything from there. I suggest you start with a simple time >>>> electrical portion with constraints to explore and we'll go from >>>> there. Shri is actually a electrical networks guy and so can speak >>>> your language. >>> >>> >>> ? I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. >>> A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). >>> i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). >>> ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space >>> (-snes_vi_type rs). >>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. >>> >>> By the way, does your problem have bounds on the variables or bounds on some function of the variables? >>> >>> Shri >>> >>> >>> >>>> >>>> Barry >>>> >>>> >>>> >>>>> >>>>> Thank you again, >>>>> >>>>> Jon >>>>> >>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>>> >>>>>> >>>>>> Jon, >>>>>> >>>>>> ?You may be in luck. In PETSc-dev >>>>>> ?http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>>> ?have now implemented variational inequality non-linear solvers >>>>>> ?with box constraints. >>>>>> >>>>>> ?That is one has a set of variables u and algebraic equations >>>>>> ?F(u) = 0 (say of size n each) but in addition one has >>>>>> ?constraints lu <= u <= uu where some or all of lu may be >>>>>> ?negative infinity (no constraints) and some or all of uu may be >>>>>> ?infinity (no constraints). There is also a constraint on the >>>>>> ?sign of F() for those equations associated with active >>>>>> ?constraints. If your constraints are not in this form sometimes >>>>>> ?you can introduce new additional variables to get it in this >>>>>> ?form. Read up a little on variational inequalities on the web. >>>>>> >>>>>> ?To use this you provide the usual SNES function and Jacobian but >>>>>> ?you also provide SNESVISetVariableBounds() there is a manual >>>>>> ?page for this function and for for SNESVI at >>>>>> ?http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>>> ?(the link is broken right now but Satish is fixing it). Plus >>>>>> ?several examples in src/snes/examples/tutorials (in petsc-dev). >>>>>> >>>>>> ?This is all new code so we would be interested in your feedback. >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am trying to add a constraint feature to my first PETSc >>>>>>> application, which uses the finite difference method to calculate >>>>>>> the potential distribution produced by a collection of electrodes >>>>>>> in a resistive medium. I would like to make this simulation more >>>>>>> realistic by imposing a maximum electric current and a maximum >>>>>>> potential difference that can be supplied to each electrode by the >>>>>>> power supply. If the medium between the electrodes is very >>>>>>> conductive, the current maximum would be exceeded by the maximum >>>>>>> potential difference, so the potential difference should be >>>>>>> decreased from maximum until it produces the maximum current. On >>>>>>> the other hand, the potential difference between the electrodes >>>>>>> should remain at maximum as long as the current remains below >>>>>>> maximum (say, for a less conductive medium). >>>>>>> >>>>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>>>> DMDA, and I developed a set of conditional expressions that >>>>>>> describe the above constraints, but one problem is that the logic >>>>>>> relies on if-then-else decisions that are made when forming the >>>>>>> function/residual and the Jacobian. Once these decisions are made, >>>>>>> of course, the conditions are not checked again until the next >>>>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>>>> to oscillate between extreme solutions to the opposing conditions >>>>>>> with each iteration, and never converges towards a reasonable >>>>>>> solution. >>>>>>> >>>>>>> Is there a better strategy for solving such problems? Does PETSc >>>>>>> offer mechanisms to aid in their solution? I would very much >>>>>>> appreciate any hints. >>>>>>> >>>>>>> Thank you for your time, >>>>>>> >>>>>>> Jon >>>>>> >>>>> >>> >> > > From bsmith at mcs.anl.gov Mon Jul 25 23:15:34 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 25 Jul 2011 23:15:34 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> Message-ID: <64E89B83-235C-4A1E-961C-A6EFFE7DEC44@mcs.anl.gov> Vijay, VI solvers do not solve (directly anyway) an optimization problem. For optimization you should use the Tao package. VI solvers solve a nonlinear equation coupled to inequality constraints. In our case the inequality constraints can only be on the independent variables (called boxed constraints). Essentially for each equation F_i(x) at the solution x* either F_i(x*) is 0 or x_i is on the constraint and F_i(x*) is of a particular sign. I am not the person to ask about good references on this topic, I know there are lots of books and papers on the topic. Barry On Jul 25, 2011, at 10:17 PM, Vijay S. Mahadevan wrote: > Barry/Shri, > > Is the new VI SNES type in general terms a constrained nonlinear > solver ? In other words, given a certain constraint as a function of > the variables of interest, does it find the optimal solution that both > minimizes the residual and satisfy the constraint ? I have been > looking at L-BFGS constrained optimization methods and from what I > understand, the variational inequality solver seems to fall under a > similar but much broader category. I would much appreciate it if you > can point me to some papers related to the method. And if I > misunderstood the domain of applicability, please do feel free to > correct me. > > Thanks, > Vijay > > On Mon, Jul 25, 2011 at 9:58 PM, Barry Smith wrote: >> >> On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: >> >>> Hi Shri, >>> >>> Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. >>> >>> For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? >>> >> >> You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. >> >> Barry >> >>> Thank you again, >>> >>> Jon >>> >>> On 2011-07-25, at 11:58 AM, Shri wrote: >>> >>>> >>>> ----- Original Message ----- >>>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>>>> >>>>>> Barry, >>>>>> >>>>>> Thank you so much for your response. Lucky, indeed! I look forward >>>>>> to trying out these new features. >>>>>> >>>>>> I neglected to mention in my original post that my electrical >>>>>> problem is part of a DAE, which includes a time-dependent heating >>>>>> problem. Can SNESVI constraints be used in conjunction with >>>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>>>>> need the constraints for the time-independent electrical portion.) >>>>> >>>>> We have not yet put that in but Shri is starting to look at that just >>>>> now. Basically we would have a TSVISetVariableBounds() and handle >>>>> everything from there. I suggest you start with a simple time >>>>> electrical portion with constraints to explore and we'll go from >>>>> there. Shri is actually a electrical networks guy and so can speak >>>>> your language. >>>> >>>> >>>> I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. >>>> A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). >>>> i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). >>>> ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space >>>> (-snes_vi_type rs). >>>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. >>>> >>>> By the way, does your problem have bounds on the variables or bounds on some function of the variables? >>>> >>>> Shri >>>> >>>> >>>> >>>>> >>>>> Barry >>>>> >>>>> >>>>> >>>>>> >>>>>> Thank you again, >>>>>> >>>>>> Jon >>>>>> >>>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>>>> >>>>>>> >>>>>>> Jon, >>>>>>> >>>>>>> You may be in luck. In PETSc-dev >>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>>>> have now implemented variational inequality non-linear solvers >>>>>>> with box constraints. >>>>>>> >>>>>>> That is one has a set of variables u and algebraic equations >>>>>>> F(u) = 0 (say of size n each) but in addition one has >>>>>>> constraints lu <= u <= uu where some or all of lu may be >>>>>>> negative infinity (no constraints) and some or all of uu may be >>>>>>> infinity (no constraints). There is also a constraint on the >>>>>>> sign of F() for those equations associated with active >>>>>>> constraints. If your constraints are not in this form sometimes >>>>>>> you can introduce new additional variables to get it in this >>>>>>> form. Read up a little on variational inequalities on the web. >>>>>>> >>>>>>> To use this you provide the usual SNES function and Jacobian but >>>>>>> you also provide SNESVISetVariableBounds() there is a manual >>>>>>> page for this function and for for SNESVI at >>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>>>> (the link is broken right now but Satish is fixing it). Plus >>>>>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>>>>> >>>>>>> This is all new code so we would be interested in your feedback. >>>>>>> >>>>>>> >>>>>>> Barry >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I am trying to add a constraint feature to my first PETSc >>>>>>>> application, which uses the finite difference method to calculate >>>>>>>> the potential distribution produced by a collection of electrodes >>>>>>>> in a resistive medium. I would like to make this simulation more >>>>>>>> realistic by imposing a maximum electric current and a maximum >>>>>>>> potential difference that can be supplied to each electrode by the >>>>>>>> power supply. If the medium between the electrodes is very >>>>>>>> conductive, the current maximum would be exceeded by the maximum >>>>>>>> potential difference, so the potential difference should be >>>>>>>> decreased from maximum until it produces the maximum current. On >>>>>>>> the other hand, the potential difference between the electrodes >>>>>>>> should remain at maximum as long as the current remains below >>>>>>>> maximum (say, for a less conductive medium). >>>>>>>> >>>>>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>>>>> DMDA, and I developed a set of conditional expressions that >>>>>>>> describe the above constraints, but one problem is that the logic >>>>>>>> relies on if-then-else decisions that are made when forming the >>>>>>>> function/residual and the Jacobian. Once these decisions are made, >>>>>>>> of course, the conditions are not checked again until the next >>>>>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>>>>> to oscillate between extreme solutions to the opposing conditions >>>>>>>> with each iteration, and never converges towards a reasonable >>>>>>>> solution. >>>>>>>> >>>>>>>> Is there a better strategy for solving such problems? Does PETSc >>>>>>>> offer mechanisms to aid in their solution? I would very much >>>>>>>> appreciate any hints. >>>>>>>> >>>>>>>> Thank you for your time, >>>>>>>> >>>>>>>> Jon >>>>>>> >>>>>> >>>> >>> >> >> From vijay.m at gmail.com Mon Jul 25 23:40:34 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Mon, 25 Jul 2011 23:40:34 -0500 Subject: [petsc-users] Conditional Constraints Message-ID: Barry, Thanks for the clarification. I didn't see a reference to any papers in the dev documentation and I probably misunderstood the usage of the variational solver. I was also going to ask whether this can be used to solve a DAE by decoupling it into a time dependent equation and an algebraic (functional) constraint associated with it. From your explanation, I feel this might again be stretching the actual usage of the method. I will search for references on the method to understand it better. Thanks, Vijay On Jul 25, 2011 11:16 PM, "Barry Smith" wrote: > > Vijay, > > VI solvers do not solve (directly anyway) an optimization problem. For optimization you should use the Tao package. VI solvers solve a nonlinear equation coupled to inequality constraints. In our case the inequality constraints can only be on the independent variables (called boxed constraints). Essentially for each equation F_i(x) at the solution x* either F_i(x*) is 0 or x_i is on the constraint and F_i(x*) is of a particular sign. I am not the person to ask about good references on this topic, I know there are lots of books and papers on the topic. > > Barry > > > > On Jul 25, 2011, at 10:17 PM, Vijay S. Mahadevan wrote: > >> Barry/Shri, >> >> Is the new VI SNES type in general terms a constrained nonlinear >> solver ? In other words, given a certain constraint as a function of >> the variables of interest, does it find the optimal solution that both >> minimizes the residual and satisfy the constraint ? I have been >> looking at L-BFGS constrained optimization methods and from what I >> understand, the variational inequality solver seems to fall under a >> similar but much broader category. I would much appreciate it if you >> can point me to some papers related to the method. And if I >> misunderstood the domain of applicability, please do feel free to >> correct me. >> >> Thanks, >> Vijay >> >> On Mon, Jul 25, 2011 at 9:58 PM, Barry Smith wrote: >>> >>> On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: >>> >>>> Hi Shri, >>>> >>>> Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. >>>> >>>> For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? >>>> >>> >>> You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. >>> >>> Barry >>> >>>> Thank you again, >>>> >>>> Jon >>>> >>>> On 2011-07-25, at 11:58 AM, Shri wrote: >>>> >>>>> >>>>> ----- Original Message ----- >>>>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>>>>> >>>>>>> Barry, >>>>>>> >>>>>>> Thank you so much for your response. Lucky, indeed! I look forward >>>>>>> to trying out these new features. >>>>>>> >>>>>>> I neglected to mention in my original post that my electrical >>>>>>> problem is part of a DAE, which includes a time-dependent heating >>>>>>> problem. Can SNESVI constraints be used in conjunction with >>>>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>>>>>> need the constraints for the time-independent electrical portion.) >>>>>> >>>>>> We have not yet put that in but Shri is starting to look at that just >>>>>> now. Basically we would have a TSVISetVariableBounds() and handle >>>>>> everything from there. I suggest you start with a simple time >>>>>> electrical portion with constraints to explore and we'll go from >>>>>> there. Shri is actually a electrical networks guy and so can speak >>>>>> your language. >>>>> >>>>> >>>>> I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. >>>>> A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). >>>>> i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). >>>>> ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space >>>>> (-snes_vi_type rs). >>>>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. >>>>> >>>>> By the way, does your problem have bounds on the variables or bounds on some function of the variables? >>>>> >>>>> Shri >>>>> >>>>> >>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thank you again, >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>>>>> >>>>>>>> >>>>>>>> Jon, >>>>>>>> >>>>>>>> You may be in luck. In PETSc-dev >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>>>>> have now implemented variational inequality non-linear solvers >>>>>>>> with box constraints. >>>>>>>> >>>>>>>> That is one has a set of variables u and algebraic equations >>>>>>>> F(u) = 0 (say of size n each) but in addition one has >>>>>>>> constraints lu <= u <= uu where some or all of lu may be >>>>>>>> negative infinity (no constraints) and some or all of uu may be >>>>>>>> infinity (no constraints). There is also a constraint on the >>>>>>>> sign of F() for those equations associated with active >>>>>>>> constraints. If your constraints are not in this form sometimes >>>>>>>> you can introduce new additional variables to get it in this >>>>>>>> form. Read up a little on variational inequalities on the web. >>>>>>>> >>>>>>>> To use this you provide the usual SNES function and Jacobian but >>>>>>>> you also provide SNESVISetVariableBounds() there is a manual >>>>>>>> page for this function and for for SNESVI at >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>>>>> (the link is broken right now but Satish is fixing it). Plus >>>>>>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>>>>>> >>>>>>>> This is all new code so we would be interested in your feedback. >>>>>>>> >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I am trying to add a constraint feature to my first PETSc >>>>>>>>> application, which uses the finite difference method to calculate >>>>>>>>> the potential distribution produced by a collection of electrodes >>>>>>>>> in a resistive medium. I would like to make this simulation more >>>>>>>>> realistic by imposing a maximum electric current and a maximum >>>>>>>>> potential difference that can be supplied to each electrode by the >>>>>>>>> power supply. If the medium between the electrodes is very >>>>>>>>> conductive, the current maximum would be exceeded by the maximum >>>>>>>>> potential difference, so the potential difference should be >>>>>>>>> decreased from maximum until it produces the maximum current. On >>>>>>>>> the other hand, the potential difference between the electrodes >>>>>>>>> should remain at maximum as long as the current remains below >>>>>>>>> maximum (say, for a less conductive medium). >>>>>>>>> >>>>>>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>>>>>> DMDA, and I developed a set of conditional expressions that >>>>>>>>> describe the above constraints, but one problem is that the logic >>>>>>>>> relies on if-then-else decisions that are made when forming the >>>>>>>>> function/residual and the Jacobian. Once these decisions are made, >>>>>>>>> of course, the conditions are not checked again until the next >>>>>>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>>>>>> to oscillate between extreme solutions to the opposing conditions >>>>>>>>> with each iteration, and never converges towards a reasonable >>>>>>>>> solution. >>>>>>>>> >>>>>>>>> Is there a better strategy for solving such problems? Does PETSc >>>>>>>>> offer mechanisms to aid in their solution? I would very much >>>>>>>>> appreciate any hints. >>>>>>>>> >>>>>>>>> Thank you for your time, >>>>>>>>> >>>>>>>>> Jon >>>>>>>> >>>>>>> >>>>> >>>> >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Mon Jul 25 23:43:29 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Mon, 25 Jul 2011 23:43:29 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: Message-ID: <48418850.162193.1311655409532.JavaMail.root@zimbra.anl.gov> ----- Original Message ----- > Barry/Shri, > > Is the new VI SNES type in general terms a constrained nonlinear > solver ? Yes, where the constraints are on the variables. Given a function f(x) with upper and lower bounds on x, the VI solver tries to compute an x* such that for each x_i one of the following conditions is satisfied f(x_i) = 0 with xl_i < x_i < xu_i f(x_i) > 0 with x_i = xl_i f(x_i) < 0 with x_i = xu_i In other words, given a certain constraint as a function of > the variables of interest, does it find the optimal solution that both > minimizes the residual and satisfy the constraint ? I would say that the VI solver is not an optimizer but rather a constrained nonlinear solver. The only constrained optimization it can do is if you provide the gradient as f(x) with lower/upper bounds on x. I have been > looking at L-BFGS constrained optimization methods and from what I > understand, the variational inequality solver seems to fall under a > similar but much broader category. I suggest you take a look at the TAO package if you are interested in optimization related problems. http://www.mcs.anl.gov/research/projects/tao/ I would much appreciate it if you > can point me to some papers related to the method. http://www.mcs.anl.gov/~tmunson/papers/semismooth.pdf And if I > misunderstood the domain of applicability, please do feel free to > correct me. > > Thanks, > Vijay > > On Mon, Jul 25, 2011 at 9:58 PM, Barry Smith > wrote: > > > > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > > > >> Hi Shri, > >> > >> Thanks for your message and all the helpful tips. If the > >> TSVISetVariableBounds() functions are available now, I would like > >> to try them as well. Is the interface essentially the same as with > >> SNESVISetVariableBounds()? I will get back to you and Barry when I > >> have had a chance to modify my application. > >> > >> For my problem, I believe it makes sense to have bounds on one of > >> the variables as well as one function of the variables. The two > >> relevant degrees of freedom are the block voltage (one for each > >> finite difference block) and the electrode voltage (one for each > >> electrode, which may be present in multiple blocks). The electrode > >> voltage should keep a constant phase while the magnitude is > >> constrained between zero and some maximum. The block voltages near > >> the electrodes depend on the electrode voltages as well as the > >> neighbouring block voltages. The electrode current for a given > >> electrode is a function of its electrode voltage and several nearby > >> block voltages, and should be constrained in magnitude between zero > >> and some maximum (and the phase unconstrained). Would I need to use > >> SNESVISetComputeVariableBounds since the electrode current is a > >> function of the other variables? Would any other provisions need to > >> be made for the block voltages, since they depend on the electrode > >> voltages? > >> > > > > ? You cannot directly make a bound on some function of other > > ? variables. Instead you need to introduce another variable that is > > ? defined to be equal to that function and put the bound on that new > > ? variable. > > > > ? Barry > > > >> Thank you again, > >> > >> Jon > >> > >> On 2011-07-25, at 11:58 AM, Shri wrote: > >> > >>> > >>> ----- Original Message ----- > >>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > >>>> > >>>>> Barry, > >>>>> > >>>>> Thank you so much for your response. Lucky, indeed! I look > >>>>> forward > >>>>> to trying out these new features. > >>>>> > >>>>> I neglected to mention in my original post that my electrical > >>>>> problem is part of a DAE, which includes a time-dependent > >>>>> heating > >>>>> problem. Can SNESVI constraints be used in conjunction with > >>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I > >>>>> only > >>>>> need the constraints for the time-independent electrical > >>>>> portion.) > >>>> > >>>> We have not yet put that in but Shri is starting to look at that > >>>> just > >>>> now. Basically we would have a TSVISetVariableBounds() and handle > >>>> everything from there. I suggest you start with a simple time > >>>> electrical portion with constraints to explore and we'll go from > >>>> there. Shri is actually a electrical networks guy and so can > >>>> speak > >>>> your language. > >>> > >>> > >>> ? I've added TSVISetVariableBounds() for setting the bounds on the > >>> ? variables using the TS object directly. > >>> A few things that i want to mention here about using the > >>> variational inequality nonlinear solver (SNESVI). > >>> i) Use the runtime option -snes_type vi or explicitly set > >>> SNESSetType(snes,SNESVI). > >>> ii) There are two tested algorithms currently available, (a) > >>> semismooth (-snes_vi_type ss) and (b) active set or reduced space > >>> (-snes_vi_type rs). > >>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials > >>> which is a time-stepping nonlinear problem with constraints on the > >>> variables. This example does not use the TS object directly but > >>> rather a time-loop is explicitly written. Another example,ex8.c, > >>> in src/snes/examples/tests/ is a minimum surface area problem > >>> which uses SNESVI. > >>> > >>> By the way, does your problem have bounds on the variables or > >>> bounds on some function of the variables? > >>> > >>> Shri > >>> > >>> > >>> > >>>> > >>>> Barry > >>>> > >>>> > >>>> > >>>>> > >>>>> Thank you again, > >>>>> > >>>>> Jon > >>>>> > >>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: > >>>>> > >>>>>> > >>>>>> Jon, > >>>>>> > >>>>>> ?You may be in luck. In PETSc-dev > >>>>>> ?http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we > >>>>>> ?have now implemented variational inequality non-linear solvers > >>>>>> ?with box constraints. > >>>>>> > >>>>>> ?That is one has a set of variables u and algebraic equations > >>>>>> ?F(u) = 0 (say of size n each) but in addition one has > >>>>>> ?constraints lu <= u <= uu where some or all of lu may be > >>>>>> ?negative infinity (no constraints) and some or all of uu may > >>>>>> ?be > >>>>>> ?infinity (no constraints). There is also a constraint on the > >>>>>> ?sign of F() for those equations associated with active > >>>>>> ?constraints. If your constraints are not in this form > >>>>>> ?sometimes > >>>>>> ?you can introduce new additional variables to get it in this > >>>>>> ?form. Read up a little on variational inequalities on the web. > >>>>>> > >>>>>> ?To use this you provide the usual SNES function and Jacobian > >>>>>> ?but > >>>>>> ?you also provide SNESVISetVariableBounds() there is a manual > >>>>>> ?page for this function and for for SNESVI at > >>>>>> ?http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html > >>>>>> ?(the link is broken right now but Satish is fixing it). Plus > >>>>>> ?several examples in src/snes/examples/tutorials (in > >>>>>> ?petsc-dev). > >>>>>> > >>>>>> ?This is all new code so we would be interested in your > >>>>>> ?feedback. > >>>>>> > >>>>>> > >>>>>> Barry > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I am trying to add a constraint feature to my first PETSc > >>>>>>> application, which uses the finite difference method to > >>>>>>> calculate > >>>>>>> the potential distribution produced by a collection of > >>>>>>> electrodes > >>>>>>> in a resistive medium. I would like to make this simulation > >>>>>>> more > >>>>>>> realistic by imposing a maximum electric current and a maximum > >>>>>>> potential difference that can be supplied to each electrode by > >>>>>>> the > >>>>>>> power supply. If the medium between the electrodes is very > >>>>>>> conductive, the current maximum would be exceeded by the > >>>>>>> maximum > >>>>>>> potential difference, so the potential difference should be > >>>>>>> decreased from maximum until it produces the maximum current. > >>>>>>> On > >>>>>>> the other hand, the potential difference between the > >>>>>>> electrodes > >>>>>>> should remain at maximum as long as the current remains below > >>>>>>> maximum (say, for a less conductive medium). > >>>>>>> > >>>>>>> I added an extra degree of freedom (the electrode voltages) to > >>>>>>> my > >>>>>>> DMDA, and I developed a set of conditional expressions that > >>>>>>> describe the above constraints, but one problem is that the > >>>>>>> logic > >>>>>>> relies on if-then-else decisions that are made when forming > >>>>>>> the > >>>>>>> function/residual and the Jacobian. Once these decisions are > >>>>>>> made, > >>>>>>> of course, the conditions are not checked again until the next > >>>>>>> function or Jacobian evaluation. The non-linear solver then > >>>>>>> tends > >>>>>>> to oscillate between extreme solutions to the opposing > >>>>>>> conditions > >>>>>>> with each iteration, and never converges towards a reasonable > >>>>>>> solution. > >>>>>>> > >>>>>>> Is there a better strategy for solving such problems? Does > >>>>>>> PETSc > >>>>>>> offer mechanisms to aid in their solution? I would very much > >>>>>>> appreciate any hints. > >>>>>>> > >>>>>>> Thank you for your time, > >>>>>>> > >>>>>>> Jon > >>>>>> > >>>>> > >>> > >> > > > > From vijay.m at gmail.com Tue Jul 26 00:02:09 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Tue, 26 Jul 2011 00:02:09 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: <48418850.162193.1311655409532.JavaMail.root@zimbra.anl.gov> References: <48418850.162193.1311655409532.JavaMail.root@zimbra.anl.gov> Message-ID: Shri, Would it rather be more appropriate to call it a constrained minimizer then ? I have looked at TAO before and hence my curiosity in the new solver. I shall look at the reference you suggested before asking any more naive questions. Thanks, Vijay On Jul 25, 2011 11:43 PM, "Shri" wrote: > > > ----- Original Message ----- >> Barry/Shri, >> >> Is the new VI SNES type in general terms a constrained nonlinear >> solver ? > Yes, where the constraints are on the variables. > > Given a function f(x) with upper and lower bounds on x, the VI solver tries to compute an x* such that for each x_i one of the following conditions is satisfied > f(x_i) = 0 with xl_i < x_i < xu_i > f(x_i) > 0 with x_i = xl_i > f(x_i) < 0 with x_i = xu_i > > In other words, given a certain constraint as a function of >> the variables of interest, does it find the optimal solution that both >> minimizes the residual and satisfy the constraint ? > > I would say that the VI solver is not an optimizer but rather a constrained nonlinear solver. > The only constrained optimization it can do is if you provide the gradient as f(x) with > lower/upper bounds on x. > > > I have been >> looking at L-BFGS constrained optimization methods and from what I >> understand, the variational inequality solver seems to fall under a >> similar but much broader category. > I suggest you take a look at the TAO package if you are interested in optimization related problems. > http://www.mcs.anl.gov/research/projects/tao/ > > > I would much appreciate it if you >> can point me to some papers related to the method. > > http://www.mcs.anl.gov/~tmunson/papers/semismooth.pdf > > And if I >> misunderstood the domain of applicability, please do feel free to >> correct me. >> >> Thanks, >> Vijay >> >> On Mon, Jul 25, 2011 at 9:58 PM, Barry Smith >> wrote: >> > >> > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: >> > >> >> Hi Shri, >> >> >> >> Thanks for your message and all the helpful tips. If the >> >> TSVISetVariableBounds() functions are available now, I would like >> >> to try them as well. Is the interface essentially the same as with >> >> SNESVISetVariableBounds()? I will get back to you and Barry when I >> >> have had a chance to modify my application. >> >> >> >> For my problem, I believe it makes sense to have bounds on one of >> >> the variables as well as one function of the variables. The two >> >> relevant degrees of freedom are the block voltage (one for each >> >> finite difference block) and the electrode voltage (one for each >> >> electrode, which may be present in multiple blocks). The electrode >> >> voltage should keep a constant phase while the magnitude is >> >> constrained between zero and some maximum. The block voltages near >> >> the electrodes depend on the electrode voltages as well as the >> >> neighbouring block voltages. The electrode current for a given >> >> electrode is a function of its electrode voltage and several nearby >> >> block voltages, and should be constrained in magnitude between zero >> >> and some maximum (and the phase unconstrained). Would I need to use >> >> SNESVISetComputeVariableBounds since the electrode current is a >> >> function of the other variables? Would any other provisions need to >> >> be made for the block voltages, since they depend on the electrode >> >> voltages? >> >> >> > >> > You cannot directly make a bound on some function of other >> > variables. Instead you need to introduce another variable that is >> > defined to be equal to that function and put the bound on that new >> > variable. >> > >> > Barry >> > >> >> Thank you again, >> >> >> >> Jon >> >> >> >> On 2011-07-25, at 11:58 AM, Shri wrote: >> >> >> >>> >> >>> ----- Original Message ----- >> >>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >> >>>> >> >>>>> Barry, >> >>>>> >> >>>>> Thank you so much for your response. Lucky, indeed! I look >> >>>>> forward >> >>>>> to trying out these new features. >> >>>>> >> >>>>> I neglected to mention in my original post that my electrical >> >>>>> problem is part of a DAE, which includes a time-dependent >> >>>>> heating >> >>>>> problem. Can SNESVI constraints be used in conjunction with >> >>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I >> >>>>> only >> >>>>> need the constraints for the time-independent electrical >> >>>>> portion.) >> >>>> >> >>>> We have not yet put that in but Shri is starting to look at that >> >>>> just >> >>>> now. Basically we would have a TSVISetVariableBounds() and handle >> >>>> everything from there. I suggest you start with a simple time >> >>>> electrical portion with constraints to explore and we'll go from >> >>>> there. Shri is actually a electrical networks guy and so can >> >>>> speak >> >>>> your language. >> >>> >> >>> >> >>> I've added TSVISetVariableBounds() for setting the bounds on the >> >>> variables using the TS object directly. >> >>> A few things that i want to mention here about using the >> >>> variational inequality nonlinear solver (SNESVI). >> >>> i) Use the runtime option -snes_type vi or explicitly set >> >>> SNESSetType(snes,SNESVI). >> >>> ii) There are two tested algorithms currently available, (a) >> >>> semismooth (-snes_vi_type ss) and (b) active set or reduced space >> >>> (-snes_vi_type rs). >> >>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials >> >>> which is a time-stepping nonlinear problem with constraints on the >> >>> variables. This example does not use the TS object directly but >> >>> rather a time-loop is explicitly written. Another example,ex8.c, >> >>> in src/snes/examples/tests/ is a minimum surface area problem >> >>> which uses SNESVI. >> >>> >> >>> By the way, does your problem have bounds on the variables or >> >>> bounds on some function of the variables? >> >>> >> >>> Shri >> >>> >> >>> >> >>> >> >>>> >> >>>> Barry >> >>>> >> >>>> >> >>>> >> >>>>> >> >>>>> Thank you again, >> >>>>> >> >>>>> Jon >> >>>>> >> >>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >> >>>>> >> >>>>>> >> >>>>>> Jon, >> >>>>>> >> >>>>>> You may be in luck. In PETSc-dev >> >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >> >>>>>> have now implemented variational inequality non-linear solvers >> >>>>>> with box constraints. >> >>>>>> >> >>>>>> That is one has a set of variables u and algebraic equations >> >>>>>> F(u) = 0 (say of size n each) but in addition one has >> >>>>>> constraints lu <= u <= uu where some or all of lu may be >> >>>>>> negative infinity (no constraints) and some or all of uu may >> >>>>>> be >> >>>>>> infinity (no constraints). There is also a constraint on the >> >>>>>> sign of F() for those equations associated with active >> >>>>>> constraints. If your constraints are not in this form >> >>>>>> sometimes >> >>>>>> you can introduce new additional variables to get it in this >> >>>>>> form. Read up a little on variational inequalities on the web. >> >>>>>> >> >>>>>> To use this you provide the usual SNES function and Jacobian >> >>>>>> but >> >>>>>> you also provide SNESVISetVariableBounds() there is a manual >> >>>>>> page for this function and for for SNESVI at >> >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >> >>>>>> (the link is broken right now but Satish is fixing it). Plus >> >>>>>> several examples in src/snes/examples/tutorials (in >> >>>>>> petsc-dev). >> >>>>>> >> >>>>>> This is all new code so we would be interested in your >> >>>>>> feedback. >> >>>>>> >> >>>>>> >> >>>>>> Barry >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >> >>>>>> >> >>>>>>> Hi, >> >>>>>>> >> >>>>>>> I am trying to add a constraint feature to my first PETSc >> >>>>>>> application, which uses the finite difference method to >> >>>>>>> calculate >> >>>>>>> the potential distribution produced by a collection of >> >>>>>>> electrodes >> >>>>>>> in a resistive medium. I would like to make this simulation >> >>>>>>> more >> >>>>>>> realistic by imposing a maximum electric current and a maximum >> >>>>>>> potential difference that can be supplied to each electrode by >> >>>>>>> the >> >>>>>>> power supply. If the medium between the electrodes is very >> >>>>>>> conductive, the current maximum would be exceeded by the >> >>>>>>> maximum >> >>>>>>> potential difference, so the potential difference should be >> >>>>>>> decreased from maximum until it produces the maximum current. >> >>>>>>> On >> >>>>>>> the other hand, the potential difference between the >> >>>>>>> electrodes >> >>>>>>> should remain at maximum as long as the current remains below >> >>>>>>> maximum (say, for a less conductive medium). >> >>>>>>> >> >>>>>>> I added an extra degree of freedom (the electrode voltages) to >> >>>>>>> my >> >>>>>>> DMDA, and I developed a set of conditional expressions that >> >>>>>>> describe the above constraints, but one problem is that the >> >>>>>>> logic >> >>>>>>> relies on if-then-else decisions that are made when forming >> >>>>>>> the >> >>>>>>> function/residual and the Jacobian. Once these decisions are >> >>>>>>> made, >> >>>>>>> of course, the conditions are not checked again until the next >> >>>>>>> function or Jacobian evaluation. The non-linear solver then >> >>>>>>> tends >> >>>>>>> to oscillate between extreme solutions to the opposing >> >>>>>>> conditions >> >>>>>>> with each iteration, and never converges towards a reasonable >> >>>>>>> solution. >> >>>>>>> >> >>>>>>> Is there a better strategy for solving such problems? Does >> >>>>>>> PETSc >> >>>>>>> offer mechanisms to aid in their solution? I would very much >> >>>>>>> appreciate any hints. >> >>>>>>> >> >>>>>>> Thank you for your time, >> >>>>>>> >> >>>>>>> Jon >> >>>>>> >> >>>>> >> >>> >> >> >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vanharen at nrg.eu Tue Jul 26 13:18:09 2011 From: vanharen at nrg.eu (Haren, S.W. van (Steven)) Date: Tue, 26 Jul 2011 20:18:09 +0200 Subject: [petsc-users] Using fortran modules in petsc References: <48418850.162193.1311655409532.JavaMail.root@zimbra.anl.gov> Message-ID: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D1D2@NRGPEX.nrgnet.intra> Dear all, I try to use fortran modules to have the same variable declarations over my main program and all my subroutines. My module file looks like: module variables #include "finclude/petscsys.h" #include "finclude/petscvec.h" #include "finclude/petscmat.h" #include "finclude/petscpc.h" #include "finclude/petscksp.h" #include "finclude/petscviewer.h" type::mesh_type PetscScalar x1,x2,y1,y2,Lx,Ly,deltax,deltay,ratiox,ratioy PetscInt Nx,Ny,Npx,Npy,Np,Nux_b,Nux_in,Nux_t,Nuy_b,Nuy_in,Nuy_t,Nu_t,Nb PetscInt Nvx_b,Nvx_in,Nvx_t,Nvy_b,Nvy_in,Nvy_t,Nv_t,N1,N2,N3,N4 end type mesh_type type(mesh_type)::mesh etc................ My main program looks like: program main use variables implicit none etc................. Then a subroutine looks like: subroutine parameters(mesh,flowParameters,ierr) use variables implicit none etc................... I have my makefile set up to first compile the module, then the main program, and then all the subroutines. The module and the main program compile just fine. However, if I try to use one of the variables from the module in the subroutine I get the following compiler error: print *,ierr 1 Error: Name 'ierr' at (1) is an ambiguous reference to 'ierr' from current program unit Any tips to resolve this issue would be very helpful. I could manually declare all the variables in the main program and all the subroutines (this is how I did it before and it works fine). However, my program is growing and for efficiency reasons I would really like to have all the variable declarations in one single file. Kind regards, Steven -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4823 bytes Desc: not available URL: From bsmith at mcs.anl.gov Tue Jul 26 14:32:26 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 26 Jul 2011 14:32:26 -0500 Subject: [petsc-users] Using fortran modules in petsc In-Reply-To: <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D1D2@NRGPEX.nrgnet.intra> References: <48418850.162193.1311655409532.JavaMail.root@zimbra.anl.gov> <1EC3BC0BF3DF4945832ECD4A46E7DE2F01A6D1D2@NRGPEX.nrgnet.intra> Message-ID: <9B6294A3-EFE7-4D9E-8A58-5D9CDCEB72A1@mcs.anl.gov> We cannot handle winmail.dat attachments. Seems likely you have ierr declared both in the module and in your routine, and the compiler cannot handle that. Since you are using ierr as an argument to your subroutines I don't think you should be declaring it in the module at all. Barry On Jul 26, 2011, at 1:18 PM, Haren, S.W. van (Steven) wrote: > Dear all, > > I try to use fortran modules to have the same variable declarations over my main program and all my subroutines. > > My module file looks like: > > module variables > #include "finclude/petscsys.h" > #include "finclude/petscvec.h" > #include "finclude/petscmat.h" > #include "finclude/petscpc.h" > #include "finclude/petscksp.h" > #include "finclude/petscviewer.h" > > type::mesh_type > PetscScalar x1,x2,y1,y2,Lx,Ly,deltax,deltay,ratiox,ratioy > PetscInt Nx,Ny,Npx,Npy,Np,Nux_b,Nux_in,Nux_t,Nuy_b,Nuy_in,Nuy_t,Nu_t,Nb > PetscInt Nvx_b,Nvx_in,Nvx_t,Nvy_b,Nvy_in,Nvy_t,Nv_t,N1,N2,N3,N4 > end type mesh_type > type(mesh_type)::mesh > > etc................ > > My main program looks like: > > program main > > > use variables > > implicit none > > etc................. > > Then a subroutine looks like: > > subroutine parameters(mesh,flowParameters,ierr) > > use variables > > implicit none > > etc................... > > I have my makefile set up to first compile the module, then the main program, and then all the subroutines. The module and the main program compile just fine. However, if I try to use one of the variables from the module in the subroutine I get the following compiler error: > > print *,ierr > 1 > Error: Name 'ierr' at (1) is an ambiguous reference to 'ierr' from current program unit > > Any tips to resolve this issue would be very helpful. I could manually declare all the variables in the main program and all the subroutines (this is how I did it before and it works fine). However, my program is growing and for efficiency reasons I would really like to have all the variable declarations in one single file. > > Kind regards, > > Steven > > > > > > > From ping.rong at tuhh.de Tue Jul 26 15:53:36 2011 From: ping.rong at tuhh.de (Ping Rong) Date: Tue, 26 Jul 2011 22:53:36 +0200 Subject: [petsc-users] use Superlu as ilu preconditioner Message-ID: <4E2F2950.2030701@tuhh.de> Dear developers, I have compiled petsc-3.1-p8 for a while. Now I would like to use superlu as an ilu preconditioner, since it offers the drop tolerance option. I have read in a thread (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) that one can run the code with option -pc_type ilu -pc_factor_mat_solver_package superlu -mat_superlu_ilu_droptol <> to setup the ilu preconditioner. I also use the option "-help |grep superlu" to check the settings. However, no matter how I change the value of -mat_superlu_ilu_droptol, I always got the same result ... -mat_superlu_lwork <0>: size of work array in bytes used by factorization (None) -mat_superlu_ilu_droptol <0.0001>: ILU_DropTol (None) -mat_superlu_ilu_filltol <0.01>: ILU_FillTol (None) ... I dont know whether I understand it correctly. But it seems to me the value of the droptol has never been changed. In that maillist thread, it was also mentioned that the dev-version is recommended, because of some bugs in the superlu interface. I have then compiled the current dev-version. but the problem is my program is based on another library (libMesh) which uses petsc as a solver package. Since some of the interfaces have been changed in the dev-version, I would not be able to compile the libMesh with petsc anymore. Is there anyway I can correct the superlu interface in the 3.1-p8 version directly? Any help will be appreciated!! Best regards -- Ping Rong, M.Sc. Hamburg University of Technology Institut of modelling and computation Denickestra?e 17 (Room 3031) 21073 Hamburg Tel.: ++49 - (0)40 42878 2749 Fax: ++49 - (0)40 42878 43533 Email: ping.rong at tuhh.de From mdbockma at ucsd.edu Tue Jul 26 16:32:52 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Tue, 26 Jul 2011 14:32:52 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse Message-ID: Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all my makefiles for me for my current project (which is written in C++). I'd like to link PETSc w/my application but I'm not sure how to do this. Suggestions? Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From amesga1 at tigers.lsu.edu Tue Jul 26 17:25:09 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Tue, 26 Jul 2011 17:25:09 -0500 Subject: [petsc-users] hdf5 directive Message-ID: Dear all, Is there a preprocessor directive to check if PETSc has been compiled with hdf5 or not ? (something like PETSC_HAS_HDF5?!!) Best, Ata From bourdin at lsu.edu Tue Jul 26 17:36:44 2011 From: bourdin at lsu.edu (Blaise Bourdin) Date: Tue, 26 Jul 2011 17:36:44 -0500 Subject: [petsc-users] hdf5 directive In-Reply-To: References: Message-ID: PETSC_HAVE_HDF5 On Jul 26, 2011, at 5:25 PM, Ataollah Mesgarnejad wrote: > Dear all, > > Is there a preprocessor directive to check if PETSc has been compiled with hdf5 or not ? (something like PETSC_HAS_HDF5?!!) > > Best, > Ata -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin From jchludzinski at gmail.com Tue Jul 26 19:11:49 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Tue, 26 Jul 2011 20:11:49 -0400 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: Message-ID: Try using Cygwin to build the PETSc libraries with VC++. This way you'll have the *.lib (vs. *.a Unix/Linux/Cygwin libraries) files you can link against. Go to the install PETSc instructions and follow the ones for Win32. ---John On Tue, Jul 26, 2011 at 5:32 PM, Matt Bockman wrote: > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all my > makefiles for me for my current project (which is written in C++). I'd like > to link PETSc w/my application but I'm not sure how to do this. > > Suggestions? > > Thanks, > Matt > From mdbockma at ucsd.edu Tue Jul 26 19:25:15 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Tue, 26 Jul 2011 17:25:15 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: Message-ID: I am using Ubuntu Linux. Matt On Tue, Jul 26, 2011 at 5:11 PM, John Chludzinski wrote: > Try using Cygwin to build the PETSc libraries with VC++. This way > you'll have the *.lib (vs. *.a Unix/Linux/Cygwin libraries) files you > can link against. Go to the install PETSc instructions and follow the > ones for Win32. > > ---John > > On Tue, Jul 26, 2011 at 5:32 PM, Matt Bockman wrote: > > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all > my > > makefiles for me for my current project (which is written in C++). I'd > like > > to link PETSc w/my application but I'm not sure how to do this. > > > > Suggestions? > > > > Thanks, > > Matt > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Tue Jul 26 19:44:24 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Tue, 26 Jul 2011 20:44:24 -0400 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: Message-ID: Does Eclipse use its own proprietary project files vs. makefiles? ---John On Tue, Jul 26, 2011 at 8:25 PM, Matt Bockman wrote: > I am using Ubuntu Linux. > > Matt > > On Tue, Jul 26, 2011 at 5:11 PM, John Chludzinski > wrote: >> >> Try using Cygwin to build the PETSc libraries with VC++. ?This way >> you'll have the *.lib (vs. *.a Unix/Linux/Cygwin libraries) files you >> can link against. ?Go to the install PETSc instructions and follow the >> ones for Win32. >> >> ---John >> >> On Tue, Jul 26, 2011 at 5:32 PM, Matt Bockman wrote: >> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all >> > my >> > makefiles for me for my current project (which is written in C++). I'd >> > like >> > to link PETSc w/my application but I'm not sure how to do this. >> > >> > Suggestions? >> > >> > Thanks, >> > Matt >> > > > From hzhang at mcs.anl.gov Tue Jul 26 21:13:51 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 26 Jul 2011 21:13:51 -0500 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: <4E2F2950.2030701@tuhh.de> References: <4E2F2950.2030701@tuhh.de> Message-ID: Did you call KSPSetFromOptions(ksp)? Run your code with '-options_table' to dump list of options inputted or '-options_left' to dump list of unused options. I tested with petsc-3.1/src/ksp/ksp/examples/tutorials/ex2.c: ./ex2 -pc_type ilu -pc_factor_mat_solver_package superlu -mat_superlu_ilu_droptol 0.1 -ksp_view ... SuperLU run parameters: ... ILU_DropTol: 0.1 ILU_FillTol: 0.01 ILU_FillFactor: 10 ILU_DropRule: 9 ILU_Norm: 2 ILU_MILU: 2 Hong On Tue, Jul 26, 2011 at 3:53 PM, Ping Rong wrote: > Dear developers, > > I have compiled petsc-3.1-p8 for a while. Now I would like to use superlu as > an ilu preconditioner, since it offers the drop tolerance option. I have > read in a thread > (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) > that one can run the code with option > > -pc_type ilu -pc_factor_mat_solver_package superlu -mat_superlu_ilu_droptol > <> > > to setup the ilu preconditioner. I also use the option "-help |grep superlu" > to check the settings. However, no matter how I change the value of > -mat_superlu_ilu_droptol, I always got the same result > ... > ?-mat_superlu_lwork <0>: size of work array in bytes used by factorization > (None) > ?-mat_superlu_ilu_droptol <0.0001>: ILU_DropTol (None) > ?-mat_superlu_ilu_filltol <0.01>: ILU_FillTol (None) > ... > I dont know whether I understand it correctly. But it seems to me the value > of the droptol has never been changed. In that maillist thread, it was also > mentioned that the dev-version is recommended, because of some bugs in the > superlu interface. I have then compiled the current dev-version. but the > problem is my program is based on another library (libMesh) which uses petsc > as a solver package. Since some of the interfaces have been changed in the > dev-version, I would not be able to compile the libMesh with petsc anymore. > Is there anyway I can correct the superlu interface in the 3.1-p8 version > directly? Any help will be appreciated!! > > Best regards > > -- > Ping Rong, M.Sc. > Hamburg University of Technology > Institut of modelling and computation > Denickestra?e 17 (Room 3031) > 21073 Hamburg > > Tel.: ++49 - (0)40 42878 2749 > Fax: ?++49 - (0)40 42878 43533 > Email: ping.rong at tuhh.de > > From bsmith at mcs.anl.gov Tue Jul 26 21:22:02 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 26 Jul 2011 21:22:02 -0500 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: Message-ID: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> There is a tiny bit of information in the PETSc users manual about Eclipse: \section{Eclipse Users} \sindex{eclipse} If you are interested in developing code that uses PETSc from Eclipse or developing PETSc in Eclipse and have knowledge of how to do indexing and build libraries in Eclipse please contact us at \trl{petsc-dev at mcs.anl.gov}. To make PETSc an Eclipse package \begin{itemize} \item Install the Mecurial plugin for Eclipse and then import the PETSc repository to Eclipse. \item elected New->Convert to C/C++ project and selected shared library. After this point you can perform searchs in the code. \end{itemize} A PETSc user has provided the following steps to build an Eclipse index for PETSc that can be used with their own code without compiling PETSc source into their project. \begin{itemize} \item In the user project source directory, create a symlink to the petsc/src directory. \item Refresh the project explorer in Eclipse, so the new symlink is followed. \item Right-click on the project in the project explorer, and choose "Index -> Rebuild". The index should now be build. \item Right-click on the PETSc symlink in the project explorer, and choose "Exclude from build..." to make sure Eclipse does not try to compile PETSc with the project. \end{itemize} We'd love to have someone figure out how to do it right and include that information. Barry On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all my makefiles for me for my current project (which is written in C++). I'd like to link PETSc w/my application but I'm not sure how to do this. > > Suggestions? > > Thanks, > Matt From mirzadeh at gmail.com Tue Jul 26 22:43:03 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Tue, 26 Jul 2011 20:43:03 -0700 Subject: [petsc-users] Compilation error in petsc 3.1-p8 with ParMetis Message-ID: Hi all, I am having trouble configuring PETSc 3.1-p8 with ParMetis 3.2 and below (I have tried all of their versions). What I can understand from the configure.log (attached below) is that for some reason PETSc cannot find the libs . I have tried the flag --download-parmetis=/path/to/tar.gz and also tried compiling ParMetis myself and using --with-parmetis-dir=/path/to/build_dir and none seem to work. In the latter case, ParMetis compiles just fine but it does not put the libs and header files under /build_dir/include or /buid_dir/lib. Does using --with-parmetis-include and --with-parmetis-lib work with multiple directory locations? I tried putting all the hear files in one directory but there are multiple header files with the same name and I do not know which one I should use. Any help is greatly appreciated. Thanks, Mohammad ================================================================================ TEST configureLibrary from PETSc.packages.ParMetis(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py:370) TESTING: configureLibrary from PETSc.packages.ParMetis(config/BuildSystem/config/package.py:370) Find an installation and check if it can work with PETSc ================================================================================== Checking for a functional ParMetis Found a copy of PARMETIS in ParMetis-3.2.0 Pushing language C Popping language C Have to rebuild ParMetis, make.inc != /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis =============================================================================== Compiling & installing Parmetis; this may take several minutes =============================================================================== sh: cd /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make clean; make lib; make minstall; make clean Executing: cd /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make clean; make lib; make minstall; make clean sh: (cd METISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' (cd ParMETISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' (cd Programs ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' rm -f *.o ;\ rm -f ../Graphs/ptest3.2.0 rm -f ../Graphs/mtest3.2.0 rm -f ../Graphs/parmetis3.2.0 rm -f ../Graphs/pometis3.2.0 make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' (cd METISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' (cd ParMETISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' (cd Programs ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' rm -f *.o ;\ rm -f ../Graphs/ptest3.2.0 rm -f ../Graphs/mtest3.2.0 rm -f ../Graphs/parmetis3.2.0 rm -f ../Graphs/pometis3.2.0 make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' ********Output of running make on ParMetis follows ******* (cd METISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' (cd ParMETISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' (cd Programs ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' rm -f *.o ;\ rm -f ../Graphs/ptest3.2.0 rm -f ../Graphs/mtest3.2.0 rm -f ../Graphs/parmetis3.2.0 rm -f ../Graphs/pometis3.2.0 make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' (cd METISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' (cd ParMETISLib ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' rm -f *.o make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' (cd Programs ; make clean ) make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' rm -f *.o ;\ rm -f ../Graphs/ptest3.2.0 rm -f ../Graphs/mtest3.2.0 rm -f ../Graphs/parmetis3.2.0 rm -f ../Graphs/pometis3.2.0 make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' make: *** No rule to make target `lib'. Stop. make: *** No rule to make target `minstall'. Stop. ********End of Output of running make on ParMetis ******* sh: cp -f /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis Executing: cp -f /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis sh: Checking for library in Download PARMETIS: ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', 'libmetis.a'] ================================================================================ TEST check from config.libraries(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/libraries.py:133) TESTING: check from config.libraries(config/BuildSystem/config/libraries.py:133) Checks that the library "libName" contains "funcs", and if it does defines HAVE_LIB"libName" - libDir may be a list of directories - libName may be a list of library names Checking for functions ['ParMETIS_V3_PartKway'] in library ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', 'libmetis.a'] ['libnsl.a', 'librt.a'] Pushing language C sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c Executing: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c sh: Pushing language C Popping language C sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl Executing: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl sh: Possible ERROR while running linker: /usr/bin/ld: cannot find -lparmetis collect2: ld returned 1 exit status output: ret = 256 error message = {/usr/bin/ld: cannot find -lparmetis collect2: ld returned 1 exit status } Pushing language C Popping language C in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char ParMETIS_V3_PartKway(); int main() { ParMETIS_V3_PartKway() ; return 0; } Popping language C ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Downloaded parmetis could not be used. Please check install in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug ******************************************************************************* File "./config/configure.py", line 257, in petsc_configure framework.configure(out = sys.stdout) File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/framework.py", line 944, in configure child.configure() File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 456, in configure self.executeTest(self.configureLibrary) File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/base.py", line 97, in executeTest ret = apply(test, args,kargs) File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 395, in configureLibrary for location, directory, lib, incl in self.generateGuesses(): File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 210, in generateGuesses raise RuntimeError('Downloaded '+self.package+' could not be used. Please check install in '+d+'\n') -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Jul 26 22:48:52 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 27 Jul 2011 03:48:52 +0000 Subject: [petsc-users] Compilation error in petsc 3.1-p8 with ParMetis In-Reply-To: References: Message-ID: On Wed, Jul 27, 2011 at 3:43 AM, Mohammad Mirzadeh wrote: > Hi all, > > I am having trouble configuring PETSc 3.1-p8 with ParMetis 3.2 and below (I > have tried all of their versions). What I can understand from the > configure.log (attached below) is that for some reason PETSc cannot find the > libs . I have tried the flag --download-parmetis=/path/to/tar.gz and also > tried compiling ParMetis myself and using > --with-parmetis-dir=/path/to/build_dir and none seem to work. In the latter > case, ParMetis compiles just fine but it does not put the libs and header > files under /build_dir/include or /buid_dir/lib. > > Does using --with-parmetis-include and --with-parmetis-lib work with > multiple directory locations? I tried putting all the hear files in one > directory but there are multiple header files with the same name and I do > not know which one I should use. Any help is greatly appreciated. > We have to fix the ParMetis build system, which is very limited. Thus you cannot download their tarball and use it. You must let us download it, or use the one from the PETSc ftp. Matt > Thanks, > Mohammad > > > > ================================================================================ > TEST configureLibrary from > PETSc.packages.ParMetis(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py:370) > TESTING: configureLibrary from > PETSc.packages.ParMetis(config/BuildSystem/config/package.py:370) > Find an installation and check if it can work with PETSc > > ================================================================================== > Checking for a functional ParMetis > Found a copy of PARMETIS in ParMetis-3.2.0 > Pushing language C > Popping language C > Have to rebuild ParMetis, make.inc != > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > > =============================================================================== > Compiling & installing Parmetis; this may take > several minutes > > =============================================================================== > > sh: cd > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make > clean; make lib; make minstall; make clean > Executing: cd > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make > clean; make lib; make minstall; make clean > sh: (cd METISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > (cd METISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > ********Output of running make on ParMetis follows ******* > (cd METISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > (cd METISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > make: *** No rule to make target `lib'. Stop. > make: *** No rule to make target `minstall'. Stop. > ********End of Output of running make on ParMetis ******* > sh: cp -f > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > Executing: cp -f > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > sh: > Checking for library in Download PARMETIS: > ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', > 'libmetis.a'] > > ================================================================================ > TEST check from > config.libraries(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/libraries.py:133) > TESTING: check from > config.libraries(config/BuildSystem/config/libraries.py:133) > Checks that the library "libName" contains "funcs", and if it does > defines HAVE_LIB"libName" > - libDir may be a list of directories > - libName may be a list of library names > Checking for functions ['ParMETIS_V3_PartKway'] in library > ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', > 'libmetis.a'] ['libnsl.a', 'librt.a'] > Pushing language C > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c > -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > Executing: > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o > conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > sh: > Pushing language C > Popping language C > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o > conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > Executing: > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o > conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > sh: > Possible ERROR while running linker: /usr/bin/ld: cannot find -lparmetis > collect2: ld returned 1 exit status > output: ret = 256 > error message = {/usr/bin/ld: cannot find -lparmetis > collect2: ld returned 1 exit status > } > Pushing language C > Popping language C > in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o > conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > Source: > #include "confdefs.h" > #include "conffix.h" > /* Override any gcc2 internal prototype to avoid an error. */ > char ParMETIS_V3_PartKway(); > > int main() { > ParMETIS_V3_PartKway() > ; > return 0; > } > Popping language C > > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > ------------------------------------------------------------------------------- > Downloaded parmetis could not be used. Please check install in > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug > > ******************************************************************************* > File "./config/configure.py", line 257, in petsc_configure > framework.configure(out = sys.stdout) > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/framework.py", > line 944, in configure > child.configure() > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 456, in configure > self.executeTest(self.configureLibrary) > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/base.py", > line 97, in executeTest > ret = apply(test, args,kargs) > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 395, in configureLibrary > for location, directory, lib, incl in self.generateGuesses(): > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 210, in generateGuesses > raise RuntimeError('Downloaded '+self.package+' could not be used. > Please check install in '+d+'\n') > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Jul 26 22:54:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 26 Jul 2011 22:54:00 -0500 Subject: [petsc-users] Compilation error in petsc 3.1-p8 with ParMetis In-Reply-To: References: Message-ID: <2F3F4A5A-1810-461E-B3FB-F8C38142B33B@mcs.anl.gov> Please do a fresh version of the --download-parmetis run and send the configure.log to petsc-maint at mcs.anl.gov This mailing list is not for large attachments or configure failures. To make sure everything is clean please remove /home/mohammad/soft/petsc-3.1-p8-debug and get a new copy of the PETSc tarball first. Note also that there is no need to attach -debug to the name of the directory since this same directory can contain two installs of PETSc by using different PETSC_ARCH names and using different ./configure options for each one. Barry On Jul 26, 2011, at 10:43 PM, Mohammad Mirzadeh wrote: > Hi all, > > I am having trouble configuring PETSc 3.1-p8 with ParMetis 3.2 and below (I have tried all of their versions). What I can understand from the configure.log (attached below) is that for some reason PETSc cannot find the libs . I have tried the flag --download-parmetis=/path/to/tar.gz and also tried compiling ParMetis myself and using --with-parmetis-dir=/path/to/build_dir and none seem to work. In the latter case, ParMetis compiles just fine but it does not put the libs and header files under /build_dir/include or /buid_dir/lib. > > Does using --with-parmetis-include and --with-parmetis-lib work with multiple directory locations? I tried putting all the hear files in one directory but there are multiple header files with the same name and I do not know which one I should use. Any help is greatly appreciated. > > Thanks, > Mohammad > > > ================================================================================ > TEST configureLibrary from PETSc.packages.ParMetis(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py:370) > TESTING: configureLibrary from PETSc.packages.ParMetis(config/BuildSystem/config/package.py:370) > Find an installation and check if it can work with PETSc > ================================================================================== > Checking for a functional ParMetis > Found a copy of PARMETIS in ParMetis-3.2.0 > Pushing language C > Popping language C > Have to rebuild ParMetis, make.inc != /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > =============================================================================== > Compiling & installing Parmetis; this may take several minutes > =============================================================================== > > sh: cd /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make clean; make lib; make minstall; make clean > Executing: cd /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make clean; make lib; make minstall; make clean > sh: (cd METISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > (cd METISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > ********Output of running make on ParMetis follows ******* > (cd METISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > (cd METISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > (cd ParMETISLib ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > rm -f *.o > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > (cd Programs ; make clean ) > make[1]: Entering directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > rm -f *.o ;\ > rm -f ../Graphs/ptest3.2.0 > rm -f ../Graphs/mtest3.2.0 > rm -f ../Graphs/parmetis3.2.0 > rm -f ../Graphs/pometis3.2.0 > make[1]: Leaving directory `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > make: *** No rule to make target `lib'. Stop. > make: *** No rule to make target `minstall'. Stop. > ********End of Output of running make on ParMetis ******* > sh: cp -f /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > Executing: cp -f /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > sh: > Checking for library in Download PARMETIS: ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', 'libmetis.a'] > ================================================================================ > TEST check from config.libraries(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/libraries.py:133) > TESTING: check from config.libraries(config/BuildSystem/config/libraries.py:133) > Checks that the library "libName" contains "funcs", and if it does defines HAVE_LIB"libName" > - libDir may be a list of directories > - libName may be a list of library names > Checking for functions ['ParMETIS_V3_PartKway'] in library ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', 'libmetis.a'] ['libnsl.a', 'librt.a'] > Pushing language C > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > Executing: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > sh: > Pushing language C > Popping language C > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > Executing: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > sh: > Possible ERROR while running linker: /usr/bin/ld: cannot find -lparmetis > collect2: ld returned 1 exit status > output: ret = 256 > error message = {/usr/bin/ld: cannot find -lparmetis > collect2: ld returned 1 exit status > } > Pushing language C > Popping language C > in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis -lmetis -lnsl -lrt -lmpichcxx -lstdc++ -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > Source: > #include "confdefs.h" > #include "conffix.h" > /* Override any gcc2 internal prototype to avoid an error. */ > char ParMETIS_V3_PartKway(); > > int main() { > ParMETIS_V3_PartKway() > ; > return 0; > } > Popping language C > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): > ------------------------------------------------------------------------------- > Downloaded parmetis could not be used. Please check install in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug > ******************************************************************************* > File "./config/configure.py", line 257, in petsc_configure > framework.configure(out = sys.stdout) > File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/framework.py", line 944, in configure > child.configure() > File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 456, in configure > self.executeTest(self.configureLibrary) > File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/base.py", line 97, in executeTest > ret = apply(test, args,kargs) > File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 395, in configureLibrary > for location, directory, lib, incl in self.generateGuesses(): > File "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", line 210, in generateGuesses > raise RuntimeError('Downloaded '+self.package+' could not be used. Please check install in '+d+'\n') From mirzadeh at gmail.com Wed Jul 27 00:58:28 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Tue, 26 Jul 2011 22:58:28 -0700 Subject: [petsc-users] Compilation error in petsc 3.1-p8 with ParMetis In-Reply-To: <2F3F4A5A-1810-461E-B3FB-F8C38142B33B@mcs.anl.gov> References: <2F3F4A5A-1810-461E-B3FB-F8C38142B33B@mcs.anl.gov> Message-ID: Matt, Barry Thank you so much for your help. The problem was solved by using --download-parmetis=1 as you pointed. Also thanks for the PETSC_ARCH tip; I was under the impression that each build needs to be in a separate directory. Thanks again, Mohammad On Tue, Jul 26, 2011 at 8:54 PM, Barry Smith wrote: > > Please do a fresh version of the --download-parmetis run and send the > configure.log to petsc-maint at mcs.anl.gov This mailing list is not for > large attachments or configure failures. > > To make sure everything is clean please remove > /home/mohammad/soft/petsc-3.1-p8-debug and get a new copy of the PETSc > tarball first. Note also that there is no need to attach -debug to the name > of the directory since this same directory can contain two installs of PETSc > by using different PETSC_ARCH names and using different ./configure options > for each one. > > > > Barry > > On Jul 26, 2011, at 10:43 PM, Mohammad Mirzadeh wrote: > > > Hi all, > > > > I am having trouble configuring PETSc 3.1-p8 with ParMetis 3.2 and below > (I have tried all of their versions). What I can understand from the > configure.log (attached below) is that for some reason PETSc cannot find the > libs . I have tried the flag --download-parmetis=/path/to/tar.gz and also > tried compiling ParMetis myself and using > --with-parmetis-dir=/path/to/build_dir and none seem to work. In the latter > case, ParMetis compiles just fine but it does not put the libs and header > files under /build_dir/include or /buid_dir/lib. > > > > Does using --with-parmetis-include and --with-parmetis-lib work with > multiple directory locations? I tried putting all the hear files in one > directory but there are multiple header files with the same name and I do > not know which one I should use. Any help is greatly appreciated. > > > > Thanks, > > Mohammad > > > > > > > ================================================================================ > > TEST configureLibrary from > PETSc.packages.ParMetis(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py:370) > > TESTING: configureLibrary from > PETSc.packages.ParMetis(config/BuildSystem/config/package.py:370) > > Find an installation and check if it can work with PETSc > > > ================================================================================== > > Checking for a functional ParMetis > > Found a copy of PARMETIS in ParMetis-3.2.0 > > Pushing language C > > Popping language C > > Have to rebuild ParMetis, make.inc != > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > > > =============================================================================== > > Compiling & installing Parmetis; this may take > several minutes > > > =============================================================================== > > > > sh: cd > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make > clean; make lib; make minstall; make clean > > Executing: cd > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0; make > clean; make lib; make minstall; make clean > > sh: (cd METISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > (cd ParMETISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > (cd Programs ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > rm -f *.o ;\ > > rm -f ../Graphs/ptest3.2.0 > > rm -f ../Graphs/mtest3.2.0 > > rm -f ../Graphs/parmetis3.2.0 > > rm -f ../Graphs/pometis3.2.0 > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > (cd METISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > (cd ParMETISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > (cd Programs ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > rm -f *.o ;\ > > rm -f ../Graphs/ptest3.2.0 > > rm -f ../Graphs/mtest3.2.0 > > rm -f ../Graphs/parmetis3.2.0 > > rm -f ../Graphs/pometis3.2.0 > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > > > ********Output of running make on ParMetis follows ******* > > (cd METISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > (cd ParMETISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > (cd Programs ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > rm -f *.o ;\ > > rm -f ../Graphs/ptest3.2.0 > > rm -f ../Graphs/mtest3.2.0 > > rm -f ../Graphs/parmetis3.2.0 > > rm -f ../Graphs/pometis3.2.0 > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > (cd METISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/METISLib' > > (cd ParMETISLib ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > rm -f *.o > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/ParMETISLib' > > (cd Programs ; make clean ) > > make[1]: Entering directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > rm -f *.o ;\ > > rm -f ../Graphs/ptest3.2.0 > > rm -f ../Graphs/mtest3.2.0 > > rm -f ../Graphs/parmetis3.2.0 > > rm -f ../Graphs/pometis3.2.0 > > make[1]: Leaving directory > `/home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/Programs' > > make: *** No rule to make target `lib'. Stop. > > make: *** No rule to make target `minstall'. Stop. > > ********End of Output of running make on ParMetis ******* > > sh: cp -f > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > > Executing: cp -f > /home/mohammad/soft/petsc-3.1-p8-debug/externalpackages/ParMetis-3.2.0/make.inc > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/conf/ParMetis > > sh: > > Checking for library in Download PARMETIS: > ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', > 'libmetis.a'] > > > ================================================================================ > > TEST check from > config.libraries(/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/libraries.py:133) > > TESTING: check from > config.libraries(config/BuildSystem/config/libraries.py:133) > > Checks that the library "libName" contains "funcs", and if it does > defines HAVE_LIB"libName" > > - libDir may be a list of directories > > - libName may be a list of library names > > Checking for functions ['ParMETIS_V3_PartKway'] in library > ['/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib/libparmetis.a', > 'libmetis.a'] ['libnsl.a', 'librt.a'] > > Pushing language C > > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c > -o conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > > Executing: > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -c -o > conftest.o -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.c > > sh: > > Pushing language C > > Popping language C > > sh: /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc > -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > Executing: > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc -o > conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > sh: > > Possible ERROR while running linker: /usr/bin/ld: cannot find -lparmetis > > collect2: ld returned 1 exit status > > output: ret = 256 > > error message = {/usr/bin/ld: cannot find -lparmetis > > collect2: ld returned 1 exit status > > } > > Pushing language C > > Popping language C > > in /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/bin/mpicc > -o conftest -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o > -Wl,-rpath,/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib -lparmetis > -lmetis -lnsl -lrt -lmpichcxx -lstdc++ > -L/home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > Source: > > #include "confdefs.h" > > #include "conffix.h" > > /* Override any gcc2 internal prototype to avoid an error. */ > > char ParMETIS_V3_PartKway(); > > > > int main() { > > ParMETIS_V3_PartKway() > > ; > > return 0; > > } > > Popping language C > > > ******************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > > ------------------------------------------------------------------------------- > > Downloaded parmetis could not be used. Please check install in > /home/mohammad/soft/petsc-3.1-p8-debug/linux-gnu-c-debug > > > ******************************************************************************* > > File "./config/configure.py", line 257, in petsc_configure > > framework.configure(out = sys.stdout) > > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/framework.py", > line 944, in configure > > child.configure() > > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 456, in configure > > self.executeTest(self.configureLibrary) > > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/base.py", > line 97, in executeTest > > ret = apply(test, args,kargs) > > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 395, in configureLibrary > > for location, directory, lib, incl in self.generateGuesses(): > > File > "/home/mohammad/soft/petsc-3.1-p8-debug/config/BuildSystem/config/package.py", > line 210, in generateGuesses > > raise RuntimeError('Downloaded '+self.package+' could not be used. > Please check install in '+d+'\n') > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Wed Jul 27 02:01:34 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 00:01:34 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: Although this is sort of orthogonal to what you do right now, I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely with PETSc(or any other library for that matter) and has excellent c/c++ support. Mohammad On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: > > There is a tiny bit of information in the PETSc users manual about > Eclipse: > > \section{Eclipse Users} \sindex{eclipse} > > If you are interested in developing code that uses PETSc from Eclipse or > developing PETSc in Eclipse and have knowledge of how to do indexing and > build libraries in Eclipse please contact us at \trl{petsc-dev at mcs.anl.gov > }. > > To make PETSc an Eclipse package > \begin{itemize} > \item Install the Mecurial plugin for Eclipse and then import the PETSc > repository to Eclipse. > \item elected New->Convert to C/C++ project and selected shared library. > After this point you can perform searchs in the code. > \end{itemize} > > A PETSc user has provided the following steps to build an Eclipse index > for PETSc that can be used with their own code without compiling PETSc > source into their project. > \begin{itemize} > \item In the user project source directory, create a symlink to the > petsc/src directory. > \item Refresh the project explorer in Eclipse, so the new symlink is > followed. > \item Right-click on the project in the project explorer, and choose "Index > -> Rebuild". The index should now be build. > \item Right-click on the PETSc symlink in the project explorer, and choose > "Exclude from build..." to make sure Eclipse does not try to compile PETSc > with the project. > \end{itemize} > > We'd love to have someone figure out how to do it right and include that > information. > > Barry > > On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all > my makefiles for me for my current project (which is written in C++). I'd > like to link PETSc w/my application but I'm not sure how to do this. > > > > Suggestions? > > > > Thanks, > > Matt > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skolb at rocketmail.com Wed Jul 27 03:22:07 2011 From: skolb at rocketmail.com (Stefan Kolb) Date: Wed, 27 Jul 2011 10:22:07 +0200 Subject: [petsc-users] Paralell matrix storage Message-ID: <201107271022.07257.skolb@rocketmail.com> Hi, I have read in the user-manual that parallel sparse matrices are stored in the manner that the first n1 rows are stored on the process with rank 0 and the next n2 rows on the process with rank 1 and so on. My question is now, is it possible to store for example the rows 0 to 24 and 50 to 74 on process with rank 0 and 25 to 49 and 75 to 99 on process with rank 1 for a 100x100 matrix with two processes. Best regards, Stefan From jchludzinski at gmail.com Wed Jul 27 04:29:58 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 27 Jul 2011 05:29:58 -0400 Subject: [petsc-users] MatCreateSeqDense ? Message-ID: I'm trying to create a dense matrices from values I'm reading from (binary) files. I tried the following code: Mat A; int n = SIZE; //4002 double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); ... MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); ierr = PetscFinalize();CHKERRQ(ierr); NOTE:*** I'm converting K to the FORTRAN column major from the C row major order before I call MatCreateSeqDense(...). This appears to work but when I try to use the 2 matrices I thus created with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real Am I creating the proper PETSc binary (canonical) format for my 2 matrices? ---John -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Wed Jul 27 05:51:53 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 27 Jul 2011 06:51:53 -0400 Subject: [petsc-users] MatCreateSeqDense ? In-Reply-To: References: Message-ID: I let the SLEPc code run for ~45 min. when it terminated with the same values I was getting using DSYGV in LAPACK. If I write code to directly call LAPACK (i.e., DSYGV), it uses ~3.93 min. What's up with this? ---John On Wed, Jul 27, 2011 at 5:29 AM, John Chludzinski wrote: > I'm trying to create a dense matrices from values I'm reading from (binary) > files. I tried the following code: > > Mat A; > int n = SIZE; //4002 > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > ... > MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > ierr = PetscFinalize();CHKERRQ(ierr); > > NOTE:*** I'm converting K to the FORTRAN column major from the C row major > order before I call MatCreateSeqDense(...). > > This appears to work but when I try to use the 2 matrices I thus created > with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real > > Am I creating the proper PETSc binary (canonical) format for my 2 matrices? > > ---John > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Wed Jul 27 06:03:54 2011 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 27 Jul 2011 13:03:54 +0200 Subject: [petsc-users] MatCreateSeqDense ? In-Reply-To: References: Message-ID: <644FABF0-AE88-4085-91D4-E3877E85A7E4@dsic.upv.es> Try running with -eps_gen_hermitian (since ex7 does not assume that the problem is symmetric). Jose El 27/07/2011, a las 12:51, John Chludzinski escribi?: > I let the SLEPc code run for ~45 min. when it terminated with the same values I was getting using DSYGV in LAPACK. > > If I write code to directly call LAPACK (i.e., DSYGV), it uses ~3.93 min. What's up with this? > > ---John > > > On Wed, Jul 27, 2011 at 5:29 AM, John Chludzinski wrote: > I'm trying to create a dense matrices from values I'm reading from (binary) files. I tried the following code: > > Mat A; > int n = SIZE; //4002 > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > ... > MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > ierr = PetscFinalize();CHKERRQ(ierr); > > NOTE:*** I'm converting K to the FORTRAN column major from the C row major order before I call MatCreateSeqDense(...). > > This appears to work but when I try to use the 2 matrices I thus created with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real > > Am I creating the proper PETSc binary (canonical) format for my 2 matrices? > > ---John > > > From knepley at gmail.com Wed Jul 27 08:01:03 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 27 Jul 2011 13:01:03 +0000 Subject: [petsc-users] Paralell matrix storage In-Reply-To: <201107271022.07257.skolb@rocketmail.com> References: <201107271022.07257.skolb@rocketmail.com> Message-ID: On Wed, Jul 27, 2011 at 8:22 AM, Stefan Kolb wrote: > Hi, > > I have read in the user-manual that parallel sparse matrices are stored in > the > manner that the first n1 rows are stored on the process with rank 0 and the > next n2 rows on the process with rank 1 and so on. My question is now, is > it > possible to store for example the rows 0 to 24 and 50 to 74 on process with > rank 0 and 25 to 49 and 75 to 99 on process with rank 1 for a 100x100 > matrix > with two processes. > No. The effect of a permutation is accomplished with a MaOrdering. Matt > Best regards, > Stefan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdbockma at ucsd.edu Wed Jul 27 11:14:17 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 09:14:17 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: Thanks Mohammad, I'll give that a shot. I use Qt Creator for some GUI applications so I am familiar with it, but I've never tried doing a non-Qt project in it. I'd really like to get Eclipse to work. Regarding the makefiles for eclipse. There are makefiles that it generates (which are for GNU make) but I think I can also manually create my makefiles. After sleeping on it, it seems like this might be the best option, unless I can figure out a way to configure eclipse to include the conf/variables and conf/rules files in the makefile. Matt On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh wrote: > Although this is sort of orthogonal to what you do right now, > I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely > with PETSc(or any other library for that matter) and has excellent c/c++ > support. > > Mohammad > > > On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: > >> >> There is a tiny bit of information in the PETSc users manual about >> Eclipse: >> >> \section{Eclipse Users} \sindex{eclipse} >> >> If you are interested in developing code that uses PETSc from Eclipse or >> developing PETSc in Eclipse and have knowledge of how to do indexing and >> build libraries in Eclipse please contact us at \ >> trl{petsc-dev at mcs.anl.gov}. >> >> To make PETSc an Eclipse package >> \begin{itemize} >> \item Install the Mecurial plugin for Eclipse and then import the PETSc >> repository to Eclipse. >> \item elected New->Convert to C/C++ project and selected shared library. >> After this point you can perform searchs in the code. >> \end{itemize} >> >> A PETSc user has provided the following steps to build an Eclipse index >> for PETSc that can be used with their own code without compiling PETSc >> source into their project. >> \begin{itemize} >> \item In the user project source directory, create a symlink to the >> petsc/src directory. >> \item Refresh the project explorer in Eclipse, so the new symlink is >> followed. >> \item Right-click on the project in the project explorer, and choose >> "Index -> Rebuild". The index should now be build. >> \item Right-click on the PETSc symlink in the project explorer, and choose >> "Exclude from build..." to make sure Eclipse does not try to compile PETSc >> with the project. >> \end{itemize} >> >> We'd love to have someone figure out how to do it right and include that >> information. >> >> Barry >> >> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> >> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all >> my makefiles for me for my current project (which is written in C++). I'd >> like to link PETSc w/my application but I'm not sure how to do this. >> > >> > Suggestions? >> > >> > Thanks, >> > Matt >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Wed Jul 27 12:21:35 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 10:21:35 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: So do you want to be able to compile PETSc with Eclipse or just point it to the library to use in your own applications? On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman wrote: > Thanks Mohammad, > > I'll give that a shot. I use Qt Creator for some GUI applications so I am > familiar with it, but I've never tried doing a non-Qt project in it. I'd > really like to get Eclipse to work. > > Regarding the makefiles for eclipse. There are makefiles that it generates > (which are for GNU make) but I think I can also manually create my > makefiles. After sleeping on it, it seems like this might be the best > option, unless I can figure out a way to configure eclipse to include the > conf/variables and conf/rules files in the makefile. > > Matt > > > On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh wrote: > >> Although this is sort of orthogonal to what you do right now, >> I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely >> with PETSc(or any other library for that matter) and has excellent c/c++ >> support. >> >> Mohammad >> >> >> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: >> >>> >>> There is a tiny bit of information in the PETSc users manual about >>> Eclipse: >>> >>> \section{Eclipse Users} \sindex{eclipse} >>> >>> If you are interested in developing code that uses PETSc from Eclipse or >>> developing PETSc in Eclipse and have knowledge of how to do indexing and >>> build libraries in Eclipse please contact us at \ >>> trl{petsc-dev at mcs.anl.gov}. >>> >>> To make PETSc an Eclipse package >>> \begin{itemize} >>> \item Install the Mecurial plugin for Eclipse and then import the PETSc >>> repository to Eclipse. >>> \item elected New->Convert to C/C++ project and selected shared library. >>> After this point you can perform searchs in the code. >>> \end{itemize} >>> >>> A PETSc user has provided the following steps to build an Eclipse index >>> for PETSc that can be used with their own code without compiling PETSc >>> source into their project. >>> \begin{itemize} >>> \item In the user project source directory, create a symlink to the >>> petsc/src directory. >>> \item Refresh the project explorer in Eclipse, so the new symlink is >>> followed. >>> \item Right-click on the project in the project explorer, and choose >>> "Index -> Rebuild". The index should now be build. >>> \item Right-click on the PETSc symlink in the project explorer, and >>> choose "Exclude from build..." to make sure Eclipse does not try to compile >>> PETSc with the project. >>> \end{itemize} >>> >>> We'd love to have someone figure out how to do it right and include that >>> information. >>> >>> Barry >>> >>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>> >>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates all >>> my makefiles for me for my current project (which is written in C++). I'd >>> like to link PETSc w/my application but I'm not sure how to do this. >>> > >>> > Suggestions? >>> > >>> > Thanks, >>> > Matt >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Wed Jul 27 12:23:34 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 10:23:34 -0700 Subject: [petsc-users] Paralell matrix storage In-Reply-To: References: <201107271022.07257.skolb@rocketmail.com> Message-ID: I assume you could also use AO ? On Wed, Jul 27, 2011 at 6:01 AM, Matthew Knepley wrote: > On Wed, Jul 27, 2011 at 8:22 AM, Stefan Kolb wrote: > >> Hi, >> >> I have read in the user-manual that parallel sparse matrices are stored in >> the >> manner that the first n1 rows are stored on the process with rank 0 and >> the >> next n2 rows on the process with rank 1 and so on. My question is now, is >> it >> possible to store for example the rows 0 to 24 and 50 to 74 on process >> with >> rank 0 and 25 to 49 and 75 to 99 on process with rank 1 for a 100x100 >> matrix >> with two processes. >> > > No. The effect of a permutation is accomplished with a MaOrdering. > > Matt > > >> Best regards, >> Stefan >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Wed Jul 27 12:28:17 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 27 Jul 2011 13:28:17 -0400 Subject: [petsc-users] MatCreateSeqDense ? In-Reply-To: <644FABF0-AE88-4085-91D4-E3877E85A7E4@dsic.upv.es> References: <644FABF0-AE88-4085-91D4-E3877E85A7E4@dsic.upv.es> Message-ID: $ time ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_type lapack -eps_smallest_real > x.out 2>&1 real 19m4.487s user 18m19.650s sys 0m1.762s ---John On Wed, Jul 27, 2011 at 7:03 AM, Jose E. Roman wrote: > Try running with -eps_gen_hermitian (since ex7 does not assume that the > problem is symmetric). > Jose > > > > El 27/07/2011, a las 12:51, John Chludzinski escribi?: > > > I let the SLEPc code run for ~45 min. when it terminated with the same > values I was getting using DSYGV in LAPACK. > > > > If I write code to directly call LAPACK (i.e., DSYGV), it uses ~3.93 min. > What's up with this? > > > > ---John > > > > > > On Wed, Jul 27, 2011 at 5:29 AM, John Chludzinski < > jchludzinski at gmail.com> wrote: > > I'm trying to create a dense matrices from values I'm reading from > (binary) files. I tried the following code: > > > > Mat A; > > int n = SIZE; //4002 > > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > > ... > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); > > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > ierr = PetscFinalize();CHKERRQ(ierr); > > > > NOTE:*** I'm converting K to the FORTRAN column major from the C row > major order before I call MatCreateSeqDense(...). > > > > This appears to work but when I try to use the 2 matrices I thus created > with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: > > > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real > > > > Am I creating the proper PETSc binary (canonical) format for my 2 > matrices? > > > > ---John > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdbockma at ucsd.edu Wed Jul 27 12:52:14 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 10:52:14 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: Just pointing it to the library would be sufficient. Matt On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh wrote: > So do you want to be able to compile PETSc with Eclipse or just point it to > the library to use in your own applications? > > > On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman wrote: > >> Thanks Mohammad, >> >> I'll give that a shot. I use Qt Creator for some GUI applications so I am >> familiar with it, but I've never tried doing a non-Qt project in it. I'd >> really like to get Eclipse to work. >> >> Regarding the makefiles for eclipse. There are makefiles that it generates >> (which are for GNU make) but I think I can also manually create my >> makefiles. After sleeping on it, it seems like this might be the best >> option, unless I can figure out a way to configure eclipse to include the >> conf/variables and conf/rules files in the makefile. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh wrote: >> >>> Although this is sort of orthogonal to what you do right now, >>> I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely >>> with PETSc(or any other library for that matter) and has excellent c/c++ >>> support. >>> >>> Mohammad >>> >>> >>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: >>> >>>> >>>> There is a tiny bit of information in the PETSc users manual about >>>> Eclipse: >>>> >>>> \section{Eclipse Users} \sindex{eclipse} >>>> >>>> If you are interested in developing code that uses PETSc from Eclipse or >>>> developing PETSc in Eclipse and have knowledge of how to do indexing and >>>> build libraries in Eclipse please contact us at \ >>>> trl{petsc-dev at mcs.anl.gov}. >>>> >>>> To make PETSc an Eclipse package >>>> \begin{itemize} >>>> \item Install the Mecurial plugin for Eclipse and then import the PETSc >>>> repository to Eclipse. >>>> \item elected New->Convert to C/C++ project and selected shared library. >>>> After this point you can perform searchs in the code. >>>> \end{itemize} >>>> >>>> A PETSc user has provided the following steps to build an Eclipse index >>>> for PETSc that can be used with their own code without compiling PETSc >>>> source into their project. >>>> \begin{itemize} >>>> \item In the user project source directory, create a symlink to the >>>> petsc/src directory. >>>> \item Refresh the project explorer in Eclipse, so the new symlink is >>>> followed. >>>> \item Right-click on the project in the project explorer, and choose >>>> "Index -> Rebuild". The index should now be build. >>>> \item Right-click on the PETSc symlink in the project explorer, and >>>> choose "Exclude from build..." to make sure Eclipse does not try to compile >>>> PETSc with the project. >>>> \end{itemize} >>>> >>>> We'd love to have someone figure out how to do it right and include that >>>> information. >>>> >>>> Barry >>>> >>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>>> >>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates >>>> all my makefiles for me for my current project (which is written in C++). >>>> I'd like to link PETSc w/my application but I'm not sure how to do this. >>>> > >>>> > Suggestions? >>>> > >>>> > Thanks, >>>> > Matt >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ping.rong at tuhh.de Wed Jul 27 13:10:31 2011 From: ping.rong at tuhh.de (Ping Rong) Date: Wed, 27 Jul 2011 20:10:31 +0200 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: References: <4E2F2950.2030701@tuhh.de> Message-ID: <4E305497.1070405@tuhh.de> Dear Hong, thank you very much for your time. Can you tell me when should KSPSetFromOptions(ksp) be called? well, libMesh has a class called "petsc_linear_solver", in its original code KSPSetFromOptions(ksp) is called while the object is created. I couldn't get any output, when I run the program with the option -pc_type ilu -pc_factor_mat_solver_package superlu -mat_superlu_ilu_droptol 0.1 -ksp_view The actual solve() function of the "petsc_linear_solver" contains the similar code as in the ex2.c ... ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), this->same_preconditioner ? SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); CHKERRABORT(libMesh::COMM_WORLD,ierr); // Set the tolerances for the iterative solver. Use the user-supplied // tolerance for the relative residual & leave the others at default values. ierr = KSPSetTolerances (_ksp, tol, PETSC_DEFAULT, PETSC_DEFAULT, max_its); CHKERRABORT(libMesh::COMM_WORLD,ierr); ierr = KSPSetFromOptions (_ksp); <-------------- I added these two lines, CHKERRABORT(libMesh::COMM_WORLD,ierr); <-------------- then I got proper output. // Solve the linear system ierr = KSPSolve (_ksp, rhs->vec(), solution->vec()); CHKERRABORT(libMesh::COMM_WORLD,ierr); .... The problem is that I have to solve a multiple frequency system, and for each frequency I have several RHS. The same ksp context is used to solve the systems repeatedly. I always call KSPSetOperators() with the option DIFFERENT_NONZERO_PATTERN for the first RHS and SAME_PRECONDITIONER for the rest, since the system matrix remains the same. For the first frequency, everything is fine, both DIFFERENT_NONZERO_PATTERN and SAME_PRECONDITIONER options. I got following output from the -ksp_view. ... PC Object: type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 1e-12 using diagonal shift to prevent zero pivot matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Matrix Object: type=seqaij, rows=2883, cols=2883 package used to perform factorization: superlu <---------------- superlu is properly set .... but when the second frequency comes up, KSPSetOperators() is called again with DIFFERENT_NONZERO_PATTERN option for the first RHS. the command line option doesn't seem to work anymore. ... PC Object: type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 1e-12 using diagonal shift to prevent zero pivot matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Matrix Object: type=seqaij, rows=2883, cols=2883 package used to perform factorization: petsc <------------ not superlu anymore ... and then for the second RHS, when it calls KSPSetOperators(ksp) in the solve() routine, petsc throws the following error message. [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Object is in wrong state! [0]PETSC ERROR: Cannot change solver matrix package after PC has been setup or used! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./PLTMainTest.opt on a linux-gnu named abe-vm by mubpr Wed Jul 27 19:41:22 2011 [0]PETSC ERROR: Libraries linked from /home/mubpr/libs/petsc/linux-gnu-acml-shared-opt/lib [0]PETSC ERROR: Configure run at Tue Jul 26 22:09:39 2011 [0]PETSC ERROR: Configure options --with-blas-lapack-lib=/home/mubpr/libs/acml/gfortran64_mp/lib/libacml_mp.so --with-blas-lapack-include=/home/mubpr/libs/acml/gfortran64_mp/include --with-scalar-type=complex --with-clanguage=c++ --with-mpi-dir=/home/mubpr/libs/mpich2 --download-superlu=yes --with-superlu=1 --download-parmetis=yes --with-parmetis=1 --download-superlu_dist=yes --with-superlu_dist=1 --download-mumps=yes --with-mumps=1 --download-blacs=yes --with-blacs=1 --download-scalapack=yes --with-scalapack=1 --download-spooles=yes --with-spooles=1 --download-umfpack=yes --with-umfpack=1 --with-debugging=no --with-shared=1 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCFactorSetMatSolverPackage_Factor() line 183 in src/ksp/pc/impls/factor/factimpl.c [0]PETSC ERROR: PCFactorSetMatSolverPackage() line 276 in src/ksp/pc/impls/factor/factor.c [0]PETSC ERROR: PCSetFromOptions_Factor() line 275 in src/ksp/pc/impls/factor/factimpl.c [0]PETSC ERROR: PCSetFromOptions_ILU() line 112 in src/ksp/pc/impls/factor/ilu/ilu.c [0]PETSC ERROR: PCSetFromOptions() line 185 in src/ksp/pc/interface/pcset.c [0]PETSC ERROR: KSPSetFromOptions() line 323 in src/ksp/ksp/interface/itcl.c [0]PETSC ERROR: User provided function() line 696 in "unknowndirectory/"src/solvers/petsc_linear_solver.C Any idea what the problem may be? I would be very grateful, if you can give me any hint. thanks alot best regards Ping Am 27.07.2011 04:13, schrieb Hong Zhang: > Did you call KSPSetFromOptions(ksp)? > Run your code with '-options_table' to dump list of options inputted > or '-options_left' to dump list of unused options. > > I tested with petsc-3.1/src/ksp/ksp/examples/tutorials/ex2.c: > ./ex2 > ... > SuperLU run parameters: > ... > ILU_DropTol: 0.1 > ILU_FillTol: 0.01 > ILU_FillFactor: 10 > ILU_DropRule: 9 > ILU_Norm: 2 > ILU_MILU: 2 > > Hong > > On Tue, Jul 26, 2011 at 3:53 PM, Ping Rong wrote: >> Dear developers, >> >> I have compiled petsc-3.1-p8 for a while. Now I would like to use superlu as >> an ilu preconditioner, since it offers the drop tolerance option. I have >> read in a thread >> (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) >> that one can run the code with option >> >> -pc_type ilu -pc_factor_mat_solver_package superlu -mat_superlu_ilu_droptol >> <> >> >> to setup the ilu preconditioner. I also use the option "-help |grep superlu" >> to check the settings. However, no matter how I change the value of >> -mat_superlu_ilu_droptol, I always got the same result >> ... >> -mat_superlu_lwork<0>: size of work array in bytes used by factorization >> (None) >> -mat_superlu_ilu_droptol<0.0001>: ILU_DropTol (None) >> -mat_superlu_ilu_filltol<0.01>: ILU_FillTol (None) >> ... >> I dont know whether I understand it correctly. But it seems to me the value >> of the droptol has never been changed. In that maillist thread, it was also >> mentioned that the dev-version is recommended, because of some bugs in the >> superlu interface. I have then compiled the current dev-version. but the >> problem is my program is based on another library (libMesh) which uses petsc >> as a solver package. Since some of the interfaces have been changed in the >> dev-version, I would not be able to compile the libMesh with petsc anymore. >> Is there anyway I can correct the superlu interface in the 3.1-p8 version >> directly? Any help will be appreciated!! >> >> Best regards >> >> -- >> Ping Rong, M.Sc. >> Hamburg University of Technology >> Institut of modelling and computation >> Denickestra?e 17 (Room 3031) >> 21073 Hamburg >> >> Tel.: ++49 - (0)40 42878 2749 >> Fax: ++49 - (0)40 42878 43533 >> Email: ping.rong at tuhh.de >> >> -- Ping Rong, M.Sc. Hamburg University of Technology Institut of modelling and computation Denickestra?e 17 (Room 3031) 21073 Hamburg Tel.: ++49 - (0)40 42878 2749 Fax: ++49 - (0)40 42878 43533 Email: ping.rong at tuhh.de From mirzadeh at gmail.com Wed Jul 27 13:18:45 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 11:18:45 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: Ok then. Now I don't have enough experience with Eclipse so I apologize beforehand if you already know these/have tried them out. If not, hopefully they can be of help. I assume there should be a way in Eclipse to give it the link lib directory. In plain makefile that's just a simple step when linking. To get all the needed linklibs for petsc, you can do make getlinklibs in the $PETSC_DIR. As for the needed include files, they are all located in $PETSC_DIR/$PETSC_ARCH/include Again, its easy to use these directories along with your makefile. I'm not sure about how you give them to Eclipse though. Hopefully this has been helpful. Best, Mohammad On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman wrote: > Just pointing it to the library would be sufficient. > > Matt > > > On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh wrote: > >> So do you want to be able to compile PETSc with Eclipse or just point it >> to the library to use in your own applications? >> >> >> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman wrote: >> >>> Thanks Mohammad, >>> >>> I'll give that a shot. I use Qt Creator for some GUI applications so I am >>> familiar with it, but I've never tried doing a non-Qt project in it. I'd >>> really like to get Eclipse to work. >>> >>> Regarding the makefiles for eclipse. There are makefiles that it >>> generates (which are for GNU make) but I think I can also manually create my >>> makefiles. After sleeping on it, it seems like this might be the best >>> option, unless I can figure out a way to configure eclipse to include the >>> conf/variables and conf/rules files in the makefile. >>> >>> Matt >>> >>> >>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh wrote: >>> >>>> Although this is sort of orthogonal to what you do right now, >>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely >>>> with PETSc(or any other library for that matter) and has excellent c/c++ >>>> support. >>>> >>>> Mohammad >>>> >>>> >>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: >>>> >>>>> >>>>> There is a tiny bit of information in the PETSc users manual about >>>>> Eclipse: >>>>> >>>>> \section{Eclipse Users} \sindex{eclipse} >>>>> >>>>> If you are interested in developing code that uses PETSc from Eclipse >>>>> or developing PETSc in Eclipse and have knowledge of how to do indexing and >>>>> build libraries in Eclipse please contact us at \ >>>>> trl{petsc-dev at mcs.anl.gov}. >>>>> >>>>> To make PETSc an Eclipse package >>>>> \begin{itemize} >>>>> \item Install the Mecurial plugin for Eclipse and then import the PETSc >>>>> repository to Eclipse. >>>>> \item elected New->Convert to C/C++ project and selected shared >>>>> library. After this point you can perform searchs in the code. >>>>> \end{itemize} >>>>> >>>>> A PETSc user has provided the following steps to build an Eclipse >>>>> index for PETSc that can be used with their own code without compiling PETSc >>>>> source into their project. >>>>> \begin{itemize} >>>>> \item In the user project source directory, create a symlink to the >>>>> petsc/src directory. >>>>> \item Refresh the project explorer in Eclipse, so the new symlink is >>>>> followed. >>>>> \item Right-click on the project in the project explorer, and choose >>>>> "Index -> Rebuild". The index should now be build. >>>>> \item Right-click on the PETSc symlink in the project explorer, and >>>>> choose "Exclude from build..." to make sure Eclipse does not try to compile >>>>> PETSc with the project. >>>>> \end{itemize} >>>>> >>>>> We'd love to have someone figure out how to do it right and include >>>>> that information. >>>>> >>>>> Barry >>>>> >>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>>>> >>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates >>>>> all my makefiles for me for my current project (which is written in C++). >>>>> I'd like to link PETSc w/my application but I'm not sure how to do this. >>>>> > >>>>> > Suggestions? >>>>> > >>>>> > Thanks, >>>>> > Matt >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Wed Jul 27 13:21:26 2011 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 27 Jul 2011 20:21:26 +0200 Subject: [petsc-users] MatCreateSeqDense ? In-Reply-To: References: <644FABF0-AE88-4085-91D4-E3877E85A7E4@dsic.upv.es> Message-ID: Don't use "time" to measure performance; instead use -log_summary or PetscGetTime for the interesting part of the computation. In this case, computing the residuals will take a lot of time. Jose El 27/07/2011, a las 19:28, John Chludzinski escribi?: > $ time ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_type lapack -eps_smallest_real > x.out 2>&1 > > real 19m4.487s > user 18m19.650s > sys 0m1.762s > > ---John > > > On Wed, Jul 27, 2011 at 7:03 AM, Jose E. Roman wrote: > Try running with -eps_gen_hermitian (since ex7 does not assume that the problem is symmetric). > Jose > > > > El 27/07/2011, a las 12:51, John Chludzinski escribi?: > > > I let the SLEPc code run for ~45 min. when it terminated with the same values I was getting using DSYGV in LAPACK. > > > > If I write code to directly call LAPACK (i.e., DSYGV), it uses ~3.93 min. What's up with this? > > > > ---John > > > > > > On Wed, Jul 27, 2011 at 5:29 AM, John Chludzinski wrote: > > I'm trying to create a dense matrices from values I'm reading from (binary) files. I tried the following code: > > > > Mat A; > > int n = SIZE; //4002 > > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > > ... > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); > > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > ierr = PetscFinalize();CHKERRQ(ierr); > > > > NOTE:*** I'm converting K to the FORTRAN column major from the C row major order before I call MatCreateSeqDense(...). > > > > This appears to work but when I try to use the 2 matrices I thus created with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: > > > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real > > > > Am I creating the proper PETSc binary (canonical) format for my 2 matrices? > > > > ---John > > > > > > > > From mirzadeh at gmail.com Wed Jul 27 13:29:36 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 11:29:36 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: I applogize for the mistake; Include files are actually located in $PETSC_DIR/include On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh wrote: > Ok then. Now I don't have enough experience with Eclipse so > I apologize beforehand if you already know these/have tried them out. If > not, hopefully they can be of help. I assume there should be a way in > Eclipse to give it the link lib directory. In plain makefile that's just a > simple step when linking. To get all the needed linklibs for petsc, you can > do > > make getlinklibs > > in the $PETSC_DIR. As for the needed include files, they are all located > in > > $PETSC_DIR/$PETSC_ARCH/include > > Again, its easy to use these directories along with your makefile. I'm not > sure about how you give them to Eclipse though. Hopefully this has been > helpful. > > Best, > Mohammad > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman wrote: > >> Just pointing it to the library would be sufficient. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh wrote: >> >>> So do you want to be able to compile PETSc with Eclipse or just point it >>> to the library to use in your own applications? >>> >>> >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman wrote: >>> >>>> Thanks Mohammad, >>>> >>>> I'll give that a shot. I use Qt Creator for some GUI applications so I >>>> am familiar with it, but I've never tried doing a non-Qt project in it. I'd >>>> really like to get Eclipse to work. >>>> >>>> Regarding the makefiles for eclipse. There are makefiles that it >>>> generates (which are for GNU make) but I think I can also manually create my >>>> makefiles. After sleeping on it, it seems like this might be the best >>>> option, unless I can figure out a way to configure eclipse to include the >>>> conf/variables and conf/rules files in the makefile. >>>> >>>> Matt >>>> >>>> >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >>> > wrote: >>>> >>>>> Although this is sort of orthogonal to what you do right now, >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely >>>>> with PETSc(or any other library for that matter) and has excellent c/c++ >>>>> support. >>>>> >>>>> Mohammad >>>>> >>>>> >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: >>>>> >>>>>> >>>>>> There is a tiny bit of information in the PETSc users manual about >>>>>> Eclipse: >>>>>> >>>>>> \section{Eclipse Users} \sindex{eclipse} >>>>>> >>>>>> If you are interested in developing code that uses PETSc from Eclipse >>>>>> or developing PETSc in Eclipse and have knowledge of how to do indexing and >>>>>> build libraries in Eclipse please contact us at \ >>>>>> trl{petsc-dev at mcs.anl.gov}. >>>>>> >>>>>> To make PETSc an Eclipse package >>>>>> \begin{itemize} >>>>>> \item Install the Mecurial plugin for Eclipse and then import the >>>>>> PETSc repository to Eclipse. >>>>>> \item elected New->Convert to C/C++ project and selected shared >>>>>> library. After this point you can perform searchs in the code. >>>>>> \end{itemize} >>>>>> >>>>>> A PETSc user has provided the following steps to build an Eclipse >>>>>> index for PETSc that can be used with their own code without compiling PETSc >>>>>> source into their project. >>>>>> \begin{itemize} >>>>>> \item In the user project source directory, create a symlink to the >>>>>> petsc/src directory. >>>>>> \item Refresh the project explorer in Eclipse, so the new symlink is >>>>>> followed. >>>>>> \item Right-click on the project in the project explorer, and choose >>>>>> "Index -> Rebuild". The index should now be build. >>>>>> \item Right-click on the PETSc symlink in the project explorer, and >>>>>> choose "Exclude from build..." to make sure Eclipse does not try to compile >>>>>> PETSc with the project. >>>>>> \end{itemize} >>>>>> >>>>>> We'd love to have someone figure out how to do it right and include >>>>>> that information. >>>>>> >>>>>> Barry >>>>>> >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>>>>> >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates >>>>>> all my makefiles for me for my current project (which is written in C++). >>>>>> I'd like to link PETSc w/my application but I'm not sure how to do this. >>>>>> > >>>>>> > Suggestions? >>>>>> > >>>>>> > Thanks, >>>>>> > Matt >>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Wed Jul 27 13:38:41 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 27 Jul 2011 13:38:41 -0500 Subject: [petsc-users] Question on re-coding ex29 Message-ID: Dear Sir/Madam, I hope you're having a nice day. I asked several questions about src/ksp/ksp/example/tutorial/ex29.c before, and learned some beginning staff of PETSc. Right now, I want to start to use it to my own program. Firstly, I want to isolate the ComputeRHS function in ex29, which means I want to put ComputeRHS to another c program file. I did that; however, some problem pop up when I compile it. /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c ex29.c: In function ComputeRHS: ex29.c:39: error: storage class specified for parameter BCType ex29.c:44: error: expected specifier-qualifier-list before BCType ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { token ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { token ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { token ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { token ex29.c:220: error: old-style parameter declarations in prototyped function definition ex29.c:220: error: expected { at end of input make: [ex29.o] Error 1 (ignored) /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 ex29.o ComputeRHS.o -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lpetsc -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl gcc: ex29.o: No such file or directory make: [ex29] Error 1 (ignored) The program and modified makefile are attached. Could you please take a look and give me some suggestions. best, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: NewEx29.zip Type: application/zip Size: 4683 bytes Desc: not available URL: From balay at mcs.anl.gov Wed Jul 27 13:52:22 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 27 Jul 2011 13:52:22 -0500 (CDT) Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: use: make getincludedirs Satish On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > I applogize for the mistake; Include files are actually located > in $PETSC_DIR/include > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh wrote: > > > Ok then. Now I don't have enough experience with Eclipse so > > I apologize beforehand if you already know these/have tried them out. If > > not, hopefully they can be of help. I assume there should be a way in > > Eclipse to give it the link lib directory. In plain makefile that's just a > > simple step when linking. To get all the needed linklibs for petsc, you can > > do > > > > make getlinklibs > > > > in the $PETSC_DIR. As for the needed include files, they are all located > > in > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > Again, its easy to use these directories along with your makefile. I'm not > > sure about how you give them to Eclipse though. Hopefully this has been > > helpful. > > > > Best, > > Mohammad > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman wrote: > > > >> Just pointing it to the library would be sufficient. > >> > >> Matt > >> > >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh wrote: > >> > >>> So do you want to be able to compile PETSc with Eclipse or just point it > >>> to the library to use in your own applications? > >>> > >>> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman wrote: > >>> > >>>> Thanks Mohammad, > >>>> > >>>> I'll give that a shot. I use Qt Creator for some GUI applications so I > >>>> am familiar with it, but I've never tried doing a non-Qt project in it. I'd > >>>> really like to get Eclipse to work. > >>>> > >>>> Regarding the makefiles for eclipse. There are makefiles that it > >>>> generates (which are for GNU make) but I think I can also manually create my > >>>> makefiles. After sleeping on it, it seems like this might be the best > >>>> option, unless I can figure out a way to configure eclipse to include the > >>>> conf/variables and conf/rules files in the makefile. > >>>> > >>>> Matt > >>>> > >>>> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >>>> > wrote: > >>>> > >>>>> Although this is sort of orthogonal to what you do right now, > >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links nicely > >>>>> with PETSc(or any other library for that matter) and has excellent c/c++ > >>>>> support. > >>>>> > >>>>> Mohammad > >>>>> > >>>>> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith wrote: > >>>>> > >>>>>> > >>>>>> There is a tiny bit of information in the PETSc users manual about > >>>>>> Eclipse: > >>>>>> > >>>>>> \section{Eclipse Users} \sindex{eclipse} > >>>>>> > >>>>>> If you are interested in developing code that uses PETSc from Eclipse > >>>>>> or developing PETSc in Eclipse and have knowledge of how to do indexing and > >>>>>> build libraries in Eclipse please contact us at \ > >>>>>> trl{petsc-dev at mcs.anl.gov}. > >>>>>> > >>>>>> To make PETSc an Eclipse package > >>>>>> \begin{itemize} > >>>>>> \item Install the Mecurial plugin for Eclipse and then import the > >>>>>> PETSc repository to Eclipse. > >>>>>> \item elected New->Convert to C/C++ project and selected shared > >>>>>> library. After this point you can perform searchs in the code. > >>>>>> \end{itemize} > >>>>>> > >>>>>> A PETSc user has provided the following steps to build an Eclipse > >>>>>> index for PETSc that can be used with their own code without compiling PETSc > >>>>>> source into their project. > >>>>>> \begin{itemize} > >>>>>> \item In the user project source directory, create a symlink to the > >>>>>> petsc/src directory. > >>>>>> \item Refresh the project explorer in Eclipse, so the new symlink is > >>>>>> followed. > >>>>>> \item Right-click on the project in the project explorer, and choose > >>>>>> "Index -> Rebuild". The index should now be build. > >>>>>> \item Right-click on the PETSc symlink in the project explorer, and > >>>>>> choose "Exclude from build..." to make sure Eclipse does not try to compile > >>>>>> PETSc with the project. > >>>>>> \end{itemize} > >>>>>> > >>>>>> We'd love to have someone figure out how to do it right and include > >>>>>> that information. > >>>>>> > >>>>>> Barry > >>>>>> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > >>>>>> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely generates > >>>>>> all my makefiles for me for my current project (which is written in C++). > >>>>>> I'd like to link PETSc w/my application but I'm not sure how to do this. > >>>>>> > > >>>>>> > Suggestions? > >>>>>> > > >>>>>> > Thanks, > >>>>>> > Matt > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > > From jchludzinski at gmail.com Wed Jul 27 14:04:46 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 27 Jul 2011 15:04:46 -0400 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: Without wading through the attached zip file, it looks like you're missing some typedef's that come from a missing include file. ---John On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei wrote: > Dear Sir/Madam, > I hope you're having a nice day. > I asked several questions about src/ksp/ksp/example/tutorial/ex29.c > before, and learned some beginning staff of PETSc. Right now, I want to > start to use it to my own program. Firstly, I want to isolate the ComputeRHS > function in ex29, which means I want to put ComputeRHS to another c program > file. I did that; however, some problem pop up when I compile it. > > /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o > ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas > -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include > -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include > -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve > -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c > ex29.c: In function ComputeRHS: > ex29.c:39: error: storage class specified for parameter BCType > ex29.c:44: error: expected specifier-qualifier-list before BCType > ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { token > ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { token > ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { token > ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { token > ex29.c:220: error: old-style parameter declarations in prototyped function > definition > ex29.c:220: error: expected { at end of input > make: [ex29.o] Error 1 (ignored) > /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o > ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include > -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include > -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve > -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c > /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -Wall > -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 > ex29.o ComputeRHS.o > -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lpetsc > -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib > -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl -lmpich > -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl > -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl > gcc: ex29.o: No such file or directory > make: [ex29] Error 1 (ignored) > > The program and modified makefile are attached. Could you please take a > look and give me some suggestions. > > best, > Alan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdbockma at ucsd.edu Wed Jul 27 14:39:10 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 12:39:10 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: I added the include directories from "make getincludedirs" and I added the line from "make getlinklib". Eclipse creates a gcc call as follows: /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include -O0 -g3 -pg -p -Wall -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -lpetsc -lX11 -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -lflapack -lfblas -lnsl -lrt -lm -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" "../SparseMatrixPetsc.c And when it is compiled I get the following: http://pastebin.com/CbRzYcZj The source file which is being compiled is: http://pastebin.com/Q85hXvnS Please have a look. I'm not quite sure what I'm doing wrong but I feel like I'm getting closer and closer to the solution. Matt On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay wrote: > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >wrote: > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have tried them out. > If > > > not, hopefully they can be of help. I assume there should be a way in > > > Eclipse to give it the link lib directory. In plain makefile that's > just a > > > simple step when linking. To get all the needed linklibs for petsc, you > can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, they are all > located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with your makefile. I'm > not > > > sure about how you give them to Eclipse though. Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh < > mirzadeh at gmail.com>wrote: > > >> > > >>> So do you want to be able to compile PETSc with Eclipse or just point > it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI applications so > I > > >>>> am familiar with it, but I've never tried doing a non-Qt project in > it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are makefiles that it > > >>>> generates (which are for GNU make) but I think I can also manually > create my > > >>>> makefiles. After sleeping on it, it seems like this might be the > best > > >>>> option, unless I can figure out a way to configure eclipse to > include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh < > mirzadeh at gmail.com > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do right now, > > >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links > nicely > > >>>>> with PETSc(or any other library for that matter) and has excellent > c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >wrote: > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc users manual > about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses PETSc from > Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge of how to do > indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov}. > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and selected shared > > >>>>>> library. After this point you can perform searchs in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to build an Eclipse > > >>>>>> index for PETSc that can be used with their own code without > compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create a symlink to > the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so the new symlink > is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project explorer, and > choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the project explorer, > and > > >>>>>> choose "Exclude from build..." to make sure Eclipse does not try > to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it right and > include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely > generates > > >>>>>> all my makefiles for me for my current project (which is written > in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm not sure how to do > this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Wed Jul 27 14:46:25 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 27 Jul 2011 14:46:25 -0500 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: Dear John, Thanks for your quick reply. I think the error message says that the BCType has some problem. However, it is originally there and I have not changed it at all. Is that because I define it in ComputeRHS again and let ex29.c include ComputeRHS.h so the BCtype is duplicated? thanks, Alan On Wed, Jul 27, 2011 at 2:04 PM, John Chludzinski wrote: > Without wading through the attached zip file, it looks like you're missing > some typedef's that come from a missing include file. ---John > > > On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei wrote: > >> Dear Sir/Madam, >> I hope you're having a nice day. >> I asked several questions about src/ksp/ksp/example/tutorial/ex29.c >> before, and learned some beginning staff of PETSc. Right now, I want to >> start to use it to my own program. Firstly, I want to isolate the ComputeRHS >> function in ex29, which means I want to put ComputeRHS to another c program >> file. I did that; however, some problem pop up when I compile it. >> >> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >> ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas >> -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c >> ex29.c: In function ComputeRHS: >> ex29.c:39: error: storage class specified for parameter BCType >> ex29.c:44: error: expected specifier-qualifier-list before BCType >> ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { token >> ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { token >> ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { token >> ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { token >> ex29.c:220: error: old-style parameter declarations in prototyped function >> definition >> ex29.c:220: error: expected { at end of input >> make: [ex29.o] Error 1 (ignored) >> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >> ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >> -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c >> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -Wall >> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 >> ex29.o ComputeRHS.o >> -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lpetsc >> -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >> -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl -lmpich >> -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl >> -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl >> gcc: ex29.o: No such file or directory >> make: [ex29] Error 1 (ignored) >> >> The program and modified makefile are attached. Could you please take a >> look and give me some suggestions. >> >> best, >> Alan >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Wed Jul 27 14:57:55 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Wed, 27 Jul 2011 15:57:55 -0400 Subject: [petsc-users] MatCreateSeqDense ? In-Reply-To: References: <644FABF0-AE88-4085-91D4-E3877E85A7E4@dsic.upv.es> Message-ID: ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- ./ex7 on a arch-cygw named LIZZYB with 1 processor, by John Wed Jul 27 15:47:39 2011 Using Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 2010 Max Max/Min Avg Total Time (sec): 1.312e+03 1.00000 1.312e+03 Objects: 2.003e+04 1.00000 2.003e+04 Flops: 2.564e+11 1.00000 2.564e+11 2.564e+11 Flops/sec: 1.955e+08 1.00000 1.955e+08 1.955e+08 Memory: 1.029e+09 1.00000 1.029e+09 MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Reductions: 2.404e+04 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.3119e+03 100.0% 2.5645e+11 100.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.002e+04 83.3% On Wed, Jul 27, 2011 at 2:21 PM, Jose E. Roman wrote: > > Don't use "time" to measure performance; instead use -log_summary or PetscGetTime for the interesting part of the computation. In this case, computing the residuals will take a lot of time. > Jose > > > > El 27/07/2011, a las 19:28, John Chludzinski escribi?: > > > $ time ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_type lapack -eps_smallest_real > x.out 2>&1 > > > > real 19m4.487s > > user 18m19.650s > > sys 0m1.762s > > > > ---John > > > > > > On Wed, Jul 27, 2011 at 7:03 AM, Jose E. Roman wrote: > > Try running with -eps_gen_hermitian (since ex7 does not assume that the problem is symmetric). > > Jose > > > > > > > > El 27/07/2011, a las 12:51, John Chludzinski escribi?: > > > > > I let the SLEPc code run for ~45 min. when it terminated with the same values I was getting using DSYGV in LAPACK. > > > > > > If I write code to directly call LAPACK (i.e., DSYGV), it uses ~3.93 min. ?What's up with this? > > > > > > ---John > > > > > > > > > On Wed, Jul 27, 2011 at 5:29 AM, John Chludzinski wrote: > > > I'm trying to create a dense matrices from values I'm reading from (binary) files. ?I tried the following code: > > > > > > Mat A; > > > int n = SIZE; //4002 > > > double *K = (double *)calloc( sizeof(double), SIZE*SIZE ); > > > ... > > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, K, &A); > > > MatView(A,PETSC_VIEWER_BINARY_(PETSC_COMM_WORLD)); > > > ierr = PetscFinalize();CHKERRQ(ierr); > > > > > > NOTE:*** I'm converting K to the FORTRAN column major from the C row major order before I call ?MatCreateSeqDense(...). > > > > > > This appears to work but when I try to use the 2 matrices I thus created with SLEPc ex7 (generalized eigenvalue problem) it never terminates, using: > > > > > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_type lapack -eps_smallest_real > > > > > > Am I creating the proper PETSc binary (canonical) format for my 2 matrices? > > > > > > ---John > > > > > > > > > > > > > > From balay at mcs.anl.gov Wed Jul 27 15:40:49 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 27 Jul 2011 15:40:49 -0500 (CDT) Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: >>>>>> /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/../../../gcrt1.o: In function `_start': (.text+0x20): undefined reference to `main' /tmp/ccEthkOE.o: In function `solveMatrix': /home/mdbockman/Documents/Research/codes/quadrilateral_FEM/quad_FEM_svn_local/quad_FEM_svn/Debug/../SparseMatrixPetsc.c:22: undefined reference to `PetscInitialize' <<<<<< Normally you get unresolved symbols when linking stuff in wrong order. i.e the order should be: SparseMatrixPetsc.o -lpetsc and not -lpetsc SparseMatrixPetsc.o Or its possible you've added the 'make getlinklib' info to the compile command - not link command. [assuming eclipse does the build in the above 2 steps.] Satish On Wed, 27 Jul 2011, Matt Bockman wrote: > I added the include directories from "make getincludedirs" and I added the > line from "make getlinklib". Eclipse creates a gcc call as follows: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 > -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl -MMD -MP > -MF"SparseMatrixPetsc.d" -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but I feel like > I'm getting closer and closer to the solution. > > Matt > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay wrote: > > > use: > > make getincludedirs > > > > Satish > > > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > > > I applogize for the mistake; Include files are actually located > > > in $PETSC_DIR/include > > > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > >wrote: > > > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > > I apologize beforehand if you already know these/have tried them out. > > If > > > > not, hopefully they can be of help. I assume there should be a way in > > > > Eclipse to give it the link lib directory. In plain makefile that's > > just a > > > > simple step when linking. To get all the needed linklibs for petsc, you > > can > > > > do > > > > > > > > make getlinklibs > > > > > > > > in the $PETSC_DIR. As for the needed include files, they are all > > located > > > > in > > > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > > > Again, its easy to use these directories along with your makefile. I'm > > not > > > > sure about how you give them to Eclipse though. Hopefully this has been > > > > helpful. > > > > > > > > Best, > > > > Mohammad > > > > > > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > wrote: > > > > > > > >> Just pointing it to the library would be sufficient. > > > >> > > > >> Matt > > > >> > > > >> > > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh < > > mirzadeh at gmail.com>wrote: > > > >> > > > >>> So do you want to be able to compile PETSc with Eclipse or just point > > it > > > >>> to the library to use in your own applications? > > > >>> > > > >>> > > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > wrote: > > > >>> > > > >>>> Thanks Mohammad, > > > >>>> > > > >>>> I'll give that a shot. I use Qt Creator for some GUI applications so > > I > > > >>>> am familiar with it, but I've never tried doing a non-Qt project in > > it. I'd > > > >>>> really like to get Eclipse to work. > > > >>>> > > > >>>> Regarding the makefiles for eclipse. There are makefiles that it > > > >>>> generates (which are for GNU make) but I think I can also manually > > create my > > > >>>> makefiles. After sleeping on it, it seems like this might be the > > best > > > >>>> option, unless I can figure out a way to configure eclipse to > > include the > > > >>>> conf/variables and conf/rules files in the makefile. > > > >>>> > > > >>>> Matt > > > >>>> > > > >>>> > > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh < > > mirzadeh at gmail.com > > > >>>> > wrote: > > > >>>> > > > >>>>> Although this is sort of orthogonal to what you do right now, > > > >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links > > nicely > > > >>>>> with PETSc(or any other library for that matter) and has excellent > > c/c++ > > > >>>>> support. > > > >>>>> > > > >>>>> Mohammad > > > >>>>> > > > >>>>> > > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > >wrote: > > > >>>>> > > > >>>>>> > > > >>>>>> There is a tiny bit of information in the PETSc users manual > > about > > > >>>>>> Eclipse: > > > >>>>>> > > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > > >>>>>> > > > >>>>>> If you are interested in developing code that uses PETSc from > > Eclipse > > > >>>>>> or developing PETSc in Eclipse and have knowledge of how to do > > indexing and > > > >>>>>> build libraries in Eclipse please contact us at \ > > > >>>>>> trl{petsc-dev at mcs.anl.gov}. > > > >>>>>> > > > >>>>>> To make PETSc an Eclipse package > > > >>>>>> \begin{itemize} > > > >>>>>> \item Install the Mecurial plugin for Eclipse and then import the > > > >>>>>> PETSc repository to Eclipse. > > > >>>>>> \item elected New->Convert to C/C++ project and selected shared > > > >>>>>> library. After this point you can perform searchs in the code. > > > >>>>>> \end{itemize} > > > >>>>>> > > > >>>>>> A PETSc user has provided the following steps to build an Eclipse > > > >>>>>> index for PETSc that can be used with their own code without > > compiling PETSc > > > >>>>>> source into their project. > > > >>>>>> \begin{itemize} > > > >>>>>> \item In the user project source directory, create a symlink to > > the > > > >>>>>> petsc/src directory. > > > >>>>>> \item Refresh the project explorer in Eclipse, so the new symlink > > is > > > >>>>>> followed. > > > >>>>>> \item Right-click on the project in the project explorer, and > > choose > > > >>>>>> "Index -> Rebuild". The index should now be build. > > > >>>>>> \item Right-click on the PETSc symlink in the project explorer, > > and > > > >>>>>> choose "Exclude from build..." to make sure Eclipse does not try > > to compile > > > >>>>>> PETSc with the project. > > > >>>>>> \end{itemize} > > > >>>>>> > > > >>>>>> We'd love to have someone figure out how to do it right and > > include > > > >>>>>> that information. > > > >>>>>> > > > >>>>>> Barry > > > >>>>>> > > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > > >>>>>> > > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely > > generates > > > >>>>>> all my makefiles for me for my current project (which is written > > in C++). > > > >>>>>> I'd like to link PETSc w/my application but I'm not sure how to do > > this. > > > >>>>>> > > > > >>>>>> > Suggestions? > > > >>>>>> > > > > >>>>>> > Thanks, > > > >>>>>> > Matt > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > > > > > > > > > > From mirzadeh at gmail.com Wed Jul 27 15:43:12 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 27 Jul 2011 13:43:12 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: There two problems(I think) in this code. 1) there is no main function in your source code. If this is the only file you are compiling, you need to change the function name to main. 2) linking should be done after object files are created. A simple g++ call would first compile the main file and then link the object to the petsc lib i.e g++ -c -I($PETSC_INCLUDE) main.cpp g++ -o main main.o $PETSC_LIBS alternatively, you could do it in a single line if you like g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS my point is you should link to petsc after compiling your own code. So wherever in Eclipse that you are seting the parameters, make sure the $PETSC_LIBS is in the linker option and not compiler. Mohammad On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman wrote: > I added the include directories from "make getincludedirs" and I added the > line from "make getlinklib". Eclipse creates a gcc call as follows: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 > -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl -MMD -MP > -MF"SparseMatrixPetsc.d" -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but I feel like > I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay wrote: > >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > >wrote: >> > >> > > Ok then. Now I don't have enough experience with Eclipse so >> > > I apologize beforehand if you already know these/have tried them out. >> If >> > > not, hopefully they can be of help. I assume there should be a way in >> > > Eclipse to give it the link lib directory. In plain makefile that's >> just a >> > > simple step when linking. To get all the needed linklibs for petsc, >> you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, they are all >> located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with your makefile. I'm >> not >> > > sure about how you give them to Eclipse though. Hopefully this has >> been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh < >> mirzadeh at gmail.com>wrote: >> > >> >> > >>> So do you want to be able to compile PETSc with Eclipse or just >> point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI applications >> so I >> > >>>> am familiar with it, but I've never tried doing a non-Qt project in >> it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are makefiles that it >> > >>>> generates (which are for GNU make) but I think I can also manually >> create my >> > >>>> makefiles. After sleeping on it, it seems like this might be the >> best >> > >>>> option, unless I can figure out a way to configure eclipse to >> include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh < >> mirzadeh at gmail.com >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links >> nicely >> > >>>>> with PETSc(or any other library for that matter) and has excellent >> c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > >wrote: >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc users manual >> about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses PETSc from >> Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge of how to do >> indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov}. >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and selected shared >> > >>>>>> library. After this point you can perform searchs in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to build an >> Eclipse >> > >>>>>> index for PETSc that can be used with their own code without >> compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create a symlink to >> the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so the new symlink >> is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project explorer, and >> choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the project explorer, >> and >> > >>>>>> choose "Exclude from build..." to make sure Eclipse does not try >> to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it right and >> include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely >> generates >> > >>>>>> all my makefiles for me for my current project (which is written >> in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm not sure how to >> do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Wed Jul 27 17:01:27 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Wed, 27 Jul 2011 15:01:27 -0700 Subject: [petsc-users] To use SuperLU as a direct solver Message-ID: Hello, I would like to use SuperLU as a direct solver for a large size problem running on multiple processors. I have write the analytical Jacobian via FormJacobianLocal(), however, I do not know how to set the direct solver. Any thoughts? Thanks a lot! Rebecca From mdbockma at ucsd.edu Wed Jul 27 17:25:56 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 15:25:56 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: Thanks everyone for the help, I was able to compile a single example in Eclipse using the provided makefile. I'm pretty new to makefiles so it's a LOT to digest. I'm now manually creating a makefile for my project in Eclipse (and I've set Eclipse up to use a makefile that I create instead of automatically generating one). Unfortunately this is a big pain but since I can't figure out how to make Eclipse automatically include a few files in the makefile I don't really have any other choices :(. Thanks again, Matt On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh wrote: > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the only file > you are compiling, you need to change the function name to main. > 2) linking should be done after object files are created. A simple g++ call > would first compile the main file and then link the object to the petsc lib > i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. So > wherever in Eclipse that you are seting the parameters, make sure the > $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman wrote: > >> I added the include directories from "make getincludedirs" and I added the >> line from "make getlinklib". Eclipse creates a gcc call as follows: >> >> /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc >> -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include >> -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include >> -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib >> -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 >> -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl -MMD -MP >> -MF"SparseMatrixPetsc.d" -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but I feel >> like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay wrote: >> >>> use: >>> make getincludedirs >>> >>> Satish >>> >>> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >>> >>> > I applogize for the mistake; Include files are actually located >>> > in $PETSC_DIR/include >>> > >>> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh < >>> mirzadeh at gmail.com>wrote: >>> > >>> > > Ok then. Now I don't have enough experience with Eclipse so >>> > > I apologize beforehand if you already know these/have tried them out. >>> If >>> > > not, hopefully they can be of help. I assume there should be a way in >>> > > Eclipse to give it the link lib directory. In plain makefile that's >>> just a >>> > > simple step when linking. To get all the needed linklibs for petsc, >>> you can >>> > > do >>> > > >>> > > make getlinklibs >>> > > >>> > > in the $PETSC_DIR. As for the needed include files, they are all >>> located >>> > > in >>> > > >>> > > $PETSC_DIR/$PETSC_ARCH/include >>> > > >>> > > Again, its easy to use these directories along with your makefile. >>> I'm not >>> > > sure about how you give them to Eclipse though. Hopefully this has >>> been >>> > > helpful. >>> > > >>> > > Best, >>> > > Mohammad >>> > > >>> > > >>> > > >>> > > >>> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >>> wrote: >>> > > >>> > >> Just pointing it to the library would be sufficient. >>> > >> >>> > >> Matt >>> > >> >>> > >> >>> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh < >>> mirzadeh at gmail.com>wrote: >>> > >> >>> > >>> So do you want to be able to compile PETSc with Eclipse or just >>> point it >>> > >>> to the library to use in your own applications? >>> > >>> >>> > >>> >>> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >>> wrote: >>> > >>> >>> > >>>> Thanks Mohammad, >>> > >>>> >>> > >>>> I'll give that a shot. I use Qt Creator for some GUI applications >>> so I >>> > >>>> am familiar with it, but I've never tried doing a non-Qt project >>> in it. I'd >>> > >>>> really like to get Eclipse to work. >>> > >>>> >>> > >>>> Regarding the makefiles for eclipse. There are makefiles that it >>> > >>>> generates (which are for GNU make) but I think I can also manually >>> create my >>> > >>>> makefiles. After sleeping on it, it seems like this might be the >>> best >>> > >>>> option, unless I can figure out a way to configure eclipse to >>> include the >>> > >>>> conf/variables and conf/rules files in the makefile. >>> > >>>> >>> > >>>> Matt >>> > >>>> >>> > >>>> >>> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh < >>> mirzadeh at gmail.com >>> > >>>> > wrote: >>> > >>>> >>> > >>>>> Although this is sort of orthogonal to what you do right now, >>> > >>>>> I recommend Qt Creator as an alternative IDE to Eclipse. It links >>> nicely >>> > >>>>> with PETSc(or any other library for that matter) and has >>> excellent c/c++ >>> > >>>>> support. >>> > >>>>> >>> > >>>>> Mohammad >>> > >>>>> >>> > >>>>> >>> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >wrote: >>> > >>>>> >>> > >>>>>> >>> > >>>>>> There is a tiny bit of information in the PETSc users manual >>> about >>> > >>>>>> Eclipse: >>> > >>>>>> >>> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >>> > >>>>>> >>> > >>>>>> If you are interested in developing code that uses PETSc from >>> Eclipse >>> > >>>>>> or developing PETSc in Eclipse and have knowledge of how to do >>> indexing and >>> > >>>>>> build libraries in Eclipse please contact us at \ >>> > >>>>>> trl{petsc-dev at mcs.anl.gov}. >>> > >>>>>> >>> > >>>>>> To make PETSc an Eclipse package >>> > >>>>>> \begin{itemize} >>> > >>>>>> \item Install the Mecurial plugin for Eclipse and then import >>> the >>> > >>>>>> PETSc repository to Eclipse. >>> > >>>>>> \item elected New->Convert to C/C++ project and selected shared >>> > >>>>>> library. After this point you can perform searchs in the code. >>> > >>>>>> \end{itemize} >>> > >>>>>> >>> > >>>>>> A PETSc user has provided the following steps to build an >>> Eclipse >>> > >>>>>> index for PETSc that can be used with their own code without >>> compiling PETSc >>> > >>>>>> source into their project. >>> > >>>>>> \begin{itemize} >>> > >>>>>> \item In the user project source directory, create a symlink to >>> the >>> > >>>>>> petsc/src directory. >>> > >>>>>> \item Refresh the project explorer in Eclipse, so the new >>> symlink is >>> > >>>>>> followed. >>> > >>>>>> \item Right-click on the project in the project explorer, and >>> choose >>> > >>>>>> "Index -> Rebuild". The index should now be build. >>> > >>>>>> \item Right-click on the PETSc symlink in the project explorer, >>> and >>> > >>>>>> choose "Exclude from build..." to make sure Eclipse does not try >>> to compile >>> > >>>>>> PETSc with the project. >>> > >>>>>> \end{itemize} >>> > >>>>>> >>> > >>>>>> We'd love to have someone figure out how to do it right and >>> include >>> > >>>>>> that information. >>> > >>>>>> >>> > >>>>>> Barry >>> > >>>>>> >>> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>> > >>>>>> >>> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? Eclipse nicely >>> generates >>> > >>>>>> all my makefiles for me for my current project (which is written >>> in C++). >>> > >>>>>> I'd like to link PETSc w/my application but I'm not sure how to >>> do this. >>> > >>>>>> > >>> > >>>>>> > Suggestions? >>> > >>>>>> > >>> > >>>>>> > Thanks, >>> > >>>>>> > Matt >>> > >>>>>> >>> > >>>>>> >>> > >>>>> >>> > >>>> >>> > >>> >>> > >> >>> > > >>> > >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Jul 27 18:10:27 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 27 Jul 2011 23:10:27 +0000 Subject: [petsc-users] To use SuperLU as a direct solver In-Reply-To: References: Message-ID: On Wed, Jul 27, 2011 at 10:01 PM, Xuefei (Rebecca) Yuan wrote: > Hello, > > I would like to use SuperLU as a direct solver for a large size problem > running on multiple processors. > > I have write the analytical Jacobian via FormJacobianLocal(), however, I do > not know how to set the direct solver. > > Any thoughts? > You need SuperlU_Dist: -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist Matt > Thanks a lot! > > Rebecca > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.van-wachem at imperial.ac.uk Wed Jul 27 18:46:10 2011 From: b.van-wachem at imperial.ac.uk (Berend van Wachem) Date: Thu, 28 Jul 2011 00:46:10 +0100 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> Message-ID: <4E30A342.90006@imperial.ac.uk> Dear Matt, I use Eclipse and have eclipse make the makefiles. It is just a matter of indicating to eclipse where the PETSc headers/libraries are to be found, so if you have a C project which needs PETSc headers and libraries: To do this, click on your managed C project with the right sided mouse button, select Properties -> C/C++ Build -> Settings Then you get a new window with on the right hand side the various setting options. Select Includes, and add the required PETSc paths. In my case I have added ${PETSC_DIR}/include ${PETSC_DIR}/${PETSC_ARCH}/include Then select "Libraries" under the header Linker and you should set the Library search path: ${PETSC_DIR}/${PETSC_ARCH}/lib and then the libraries, in my case: m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, rt,gcc_s, pthread, X11 (you can find these easily in $PETSC_DIR/$PETSC_ARCH/petscmachineinfo.h) The nice thing is that in eclipse you can easily switch between Debug/Release code, traverse into the PETSc source code etc. It's really a very productive tool with PETSc I've found. Let me know if you have any questions. Kind regards, Berend. On 07/27/2011 11:25 PM, Matt Bockman wrote: > Thanks everyone for the help, > > I was able to compile a single example in Eclipse using the provided > makefile. I'm pretty new to makefiles so it's a LOT to digest. I'm now > manually creating a makefile for my project in Eclipse (and I've set > Eclipse up to use a makefile that I create instead of automatically > generating one). Unfortunately this is a big pain but since I can't > figure out how to make Eclipse automatically include a few files in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the > only file you are compiling, you need to change the function name to > main. > 2) linking should be done after object files are created. A simple > g++ call would first compile the main file and then link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. > So wherever in Eclipse that you are seting the parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > wrote: > > I added the include directories from "make getincludedirs" and I > added the line from "make getlinklib". Eclipse creates a gcc > call as follows: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but > I feel like I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > >wrote: > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > >wrote: > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems like this > might be the best > > >>>> option, unless I can figure out a way to configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > >wrote: > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > }. > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > From mdbockma at ucsd.edu Wed Jul 27 19:34:28 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 17:34:28 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <4E30A342.90006@imperial.ac.uk> References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> <4E30A342.90006@imperial.ac.uk> Message-ID: Thanks Berend for your thorough response, I have done what you have said but I still get the same error regarding "undefined references to PetscInitialize" etc. It's like I didn't include the petscksp.h file, but it's there. I even tried petsc.h to no avail. I'm not sure what the compiler is referring to when it says "Undefined references to ...". What I think this is is in the assembly code generated by the compiler, there is a PetscInitialize symbol that isn't found in the library. But I'm soooooooooo confused at this point :(. How did you guys all learn how to compile this? Matt On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem < b.van-wachem at imperial.ac.uk> wrote: > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where the PETSc > headers/libraries are to be found, so if you have a C project which needs > PETSc headers and libraries: > > To do this, click on your managed C project with the right sided mouse > button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side the various setting > options. > > Select Includes, and add the required PETSc paths. In my case I have added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/**include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, rt,gcc_s, > pthread, X11 > (you can find these easily in $PETSC_DIR/$PETSC_ARCH/**petscmachineinfo.h) > > The nice thing is that in eclipse you can easily switch between > Debug/Release code, traverse into the PETSc source code etc. It's really a > very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > >> Thanks everyone for the help, >> >> I was able to compile a single example in Eclipse using the provided >> makefile. I'm pretty new to makefiles so it's a LOT to digest. I'm now >> manually creating a makefile for my project in Eclipse (and I've set >> Eclipse up to use a makefile that I create instead of automatically >> generating one). Unfortunately this is a big pain but since I can't >> figure out how to make Eclipse automatically include a few files in the >> makefile I don't really have any other choices :(. >> >> Thanks again, >> Matt >> >> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > wrote: >> >> There two problems(I think) in this code. >> >> 1) there is no main function in your source code. If this is the >> only file you are compiling, you need to change the function name to >> main. >> 2) linking should be done after object files are created. A simple >> g++ call would first compile the main file and then link the object >> to the petsc lib i.e >> >> g++ -c -I($PETSC_INCLUDE) main.cpp >> g++ -o main main.o $PETSC_LIBS >> >> alternatively, you could do it in a single line if you like >> >> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >> >> my point is you should link to petsc after compiling your own code. >> So wherever in Eclipse that you are seting the parameters, make sure >> the $PETSC_LIBS is in the linker option and not compiler. >> >> Mohammad >> >> >> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > wrote: >> >> I added the include directories from "make getincludedirs" and I >> added the line from "make getlinklib". Eclipse creates a gcc >> call as follows: >> >> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/bin/**mpicc >> -I/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/include >> -I/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/include >> -I/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/**Documents/Research/codes/** >> petsc/petsc-3.1-p8/linux-gnu-**c-debug/lib >> -Wl,-rpath,/home/mdbockman/**Documents/Research/codes/** >> petsc/petsc-3.1-p8/linux-gnu-**c-debug/lib >> -L/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/**Documents/Research/codes/** >> petsc/petsc-3.1-p8/linux-gnu-**c-debug/lib >> -L/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/lib >> -L/usr/lib/x86_64-linux-gnu/**gcc/x86_64-linux-gnu/4.5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but >> I feel like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >> > wrote: >> >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually >> located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >> >**wrote: >> >> > >> > > Ok then. Now I don't have enough experience with Eclipse >> so >> > > I apologize beforehand if you already know these/have >> tried them out. If >> > > not, hopefully they can be of help. I assume there >> should be a way in >> > > Eclipse to give it the link lib directory. In plain >> makefile that's just a >> > > simple step when linking. To get all the needed >> linklibs for petsc, you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, >> they are all located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with >> your makefile. I'm not >> > > sure about how you give them to Eclipse though. >> Hopefully this has been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> > wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >> >**wrote: >> >> > >> >> > >>> So do you want to be able to compile PETSc with >> Eclipse or just point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> > wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI >> applications so I >> > >>>> am familiar with it, but I've never tried doing a >> non-Qt project in it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are >> makefiles that it >> > >>>> generates (which are for GNU make) but I think I can >> also manually create my >> > >>>> makefiles. After sleeping on it, it seems like this >> might be the best >> > >>>> option, unless I can figure out a way to configure >> eclipse to include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >> >> >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do >> right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to >> Eclipse. It links nicely >> > >>>>> with PETSc(or any other library for that matter) >> and has excellent c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >**wrote: >> >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc >> users manual about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses >> PETSc from Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge >> of how to do indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov >> >> >}. >> >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and >> then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and >> selected shared >> > >>>>>> library. After this point you can perform searchs >> in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to >> build an Eclipse >> > >>>>>> index for PETSc that can be used with their own >> code without compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create >> a symlink to the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so >> the new symlink is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project >> explorer, and choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the >> project explorer, and >> > >>>>>> choose "Exclude from build..." to make sure >> Eclipse does not try to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it >> right and include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >> Eclipse nicely generates >> > >>>>>> all my makefiles for me for my current project >> (which is written in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm >> not sure how to do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.van-wachem at imperial.ac.uk Wed Jul 27 19:39:05 2011 From: b.van-wachem at imperial.ac.uk (Berend van Wachem) Date: Thu, 28 Jul 2011 01:39:05 +0100 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> <4E30A342.90006@imperial.ac.uk> Message-ID: <4E30AFA9.8060500@imperial.ac.uk> Dear Matt, Does it say this during the compiling? Or linking? If it says this during the compiling, it means that eclipse cannot find the PETSc header files. So, it must be the setting of the "Includes". You might want to "hard-code" the directory, just to make sure. It is indeed not completely straightforward - eclipse has so many options. But trust me - many of them you will really learn to appreciate over time. Kind regards, Berend. On 07/28/2011 01:34 AM, Matt Bockman wrote: > Thanks Berend for your thorough response, > > I have done what you have said but I still get the same error regarding > "undefined references to PetscInitialize" etc. It's like I didn't > include the petscksp.h file, but it's there. I even tried petsc.h to no > avail. > > I'm not sure what the compiler is referring to when it says "Undefined > references to ...". What I think this is is in the assembly code > generated by the compiler, there is a PetscInitialize symbol that isn't > found in the library. But I'm soooooooooo confused at this point :(. How > did you guys all learn how to compile this? > > Matt > > On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem > > wrote: > > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where the PETSc > headers/libraries are to be found, so if you have a C project which > needs PETSc headers and libraries: > > To do this, click on your managed C project with the right sided > mouse button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side the various > setting options. > > Select Includes, and add the required PETSc paths. In my case I have > added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/__include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, > rt,gcc_s, pthread, X11 > (you can find these easily in > $PETSC_DIR/$PETSC_ARCH/__petscmachineinfo.h) > > The nice thing is that in eclipse you can easily switch between > Debug/Release code, traverse into the PETSc source code etc. It's > really a very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > > Thanks everyone for the help, > > I was able to compile a single example in Eclipse using the provided > makefile. I'm pretty new to makefiles so it's a LOT to digest. > I'm now > manually creating a makefile for my project in Eclipse (and I've set > Eclipse up to use a makefile that I create instead of automatically > generating one). Unfortunately this is a big pain but since I can't > figure out how to make Eclipse automatically include a few files > in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > >> wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the > only file you are compiling, you need to change the function name to > main. > 2) linking should be done after object files are created. A simple > g++ call would first compile the main file and then link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. > So wherever in Eclipse that you are seting the parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > >> wrote: > > I added the include directories from "make getincludedirs" and I > added the line from "make getlinklib". Eclipse creates a gcc > call as follows: > > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/bin/__mpicc > -I/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/include > -I/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/__Documents/Research/codes/__petsc/petsc-3.1-p8/linux-gnu-__c-debug/lib > -Wl,-rpath,/home/mdbockman/__Documents/Research/codes/__petsc/petsc-3.1-p8/linux-gnu-__c-debug/lib > -L/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/__Documents/Research/codes/__petsc/petsc-3.1-p8/linux-gnu-__c-debug/lib > -L/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/__gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but > I feel like I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > >> wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > > >>__wrote: > > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > >> wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > > >>__wrote: > > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > >> wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems like this > might be the best > > >>>> option, unless I can figure out a way to configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh > > > > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > > >>__wrote: > > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > > >}. > > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > From mdbockma at ucsd.edu Wed Jul 27 19:49:30 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 17:49:30 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <4E30AFA9.8060500@imperial.ac.uk> References: <43EAF77C-D463-480A-B228-C52753EC4DD3@mcs.anl.gov> <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> Message-ID: Hi Berend, Here is the complete output of the build: http://pastebin.com/Es4ms4EF and by the 2nd to last line "collect2: ld returned 1 exit status" I believe it is failing during the linking step. I'm not 100% sure though. Thanks for your quick response and help. Please advise, Matt On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem < b.van-wachem at imperial.ac.uk> wrote: > Dear Matt, > > Does it say this during the compiling? Or linking? > > If it says this during the compiling, it means that eclipse cannot find the > PETSc header files. So, it must be the setting of the "Includes". You might > want to "hard-code" the directory, just to make sure. > > It is indeed not completely straightforward - eclipse has so many options. > But trust me - many of them you will really learn to appreciate over time. > > Kind regards, > > Berend. > > > > On 07/28/2011 01:34 AM, Matt Bockman wrote: > >> Thanks Berend for your thorough response, >> >> I have done what you have said but I still get the same error regarding >> "undefined references to PetscInitialize" etc. It's like I didn't >> include the petscksp.h file, but it's there. I even tried petsc.h to no >> avail. >> >> I'm not sure what the compiler is referring to when it says "Undefined >> references to ...". What I think this is is in the assembly code >> generated by the compiler, there is a PetscInitialize symbol that isn't >> found in the library. But I'm soooooooooo confused at this point :(. How >> did you guys all learn how to compile this? >> >> Matt >> >> On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem >> >> >> wrote: >> >> Dear Matt, >> >> I use Eclipse and have eclipse make the makefiles. >> It is just a matter of indicating to eclipse where the PETSc >> headers/libraries are to be found, so if you have a C project which >> needs PETSc headers and libraries: >> >> To do this, click on your managed C project with the right sided >> mouse button, select >> >> Properties -> C/C++ Build -> Settings >> >> Then you get a new window with on the right hand side the various >> setting options. >> >> Select Includes, and add the required PETSc paths. In my case I have >> added >> ${PETSC_DIR}/include >> ${PETSC_DIR}/${PETSC_ARCH}/__**include >> >> Then select "Libraries" under the header Linker >> and you should set the Library search path: >> ${PETSC_DIR}/${PETSC_ARCH}/lib >> >> and then the libraries, in my case: >> m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, >> rt,gcc_s, pthread, X11 >> (you can find these easily in >> $PETSC_DIR/$PETSC_ARCH/__**petscmachineinfo.h) >> >> The nice thing is that in eclipse you can easily switch between >> Debug/Release code, traverse into the PETSc source code etc. It's >> really a very productive tool with PETSc I've found. >> >> Let me know if you have any questions. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/27/2011 11:25 PM, Matt Bockman wrote: >> >> Thanks everyone for the help, >> >> I was able to compile a single example in Eclipse using the >> provided >> makefile. I'm pretty new to makefiles so it's a LOT to digest. >> I'm now >> manually creating a makefile for my project in Eclipse (and I've >> set >> Eclipse up to use a makefile that I create instead of automatically >> generating one). Unfortunately this is a big pain but since I can't >> figure out how to make Eclipse automatically include a few files >> in the >> makefile I don't really have any other choices :(. >> >> Thanks again, >> Matt >> >> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh >> >> >> wrote: >> >> There two problems(I think) in this code. >> >> 1) there is no main function in your source code. If this is the >> only file you are compiling, you need to change the function name >> to >> main. >> 2) linking should be done after object files are created. A simple >> g++ call would first compile the main file and then link the object >> to the petsc lib i.e >> >> g++ -c -I($PETSC_INCLUDE) main.cpp >> g++ -o main main.o $PETSC_LIBS >> >> alternatively, you could do it in a single line if you like >> >> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >> >> my point is you should link to petsc after compiling your own code. >> So wherever in Eclipse that you are seting the parameters, make >> sure >> the $PETSC_LIBS is in the linker option and not compiler. >> >> Mohammad >> >> >> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman >> >> >> wrote: >> >> I added the include directories from "make getincludedirs" and I >> added the line from "make getlinklib". Eclipse creates a gcc >> call as follows: >> >> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/bin/__**mpicc >> -I/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/**include >> -I/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/include >> -I/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/**include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/__**Documents/Research/codes/__** >> petsc/petsc-3.1-p8/linux-gnu-_**_c-debug/lib >> -Wl,-rpath,/home/mdbockman/__**Documents/Research/codes/__** >> petsc/petsc-3.1-p8/linux-gnu-_**_c-debug/lib >> -L/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/__**Documents/Research/codes/__** >> petsc/petsc-3.1-p8/linux-gnu-_**_c-debug/lib >> -L/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/lib >> -L/usr/lib/x86_64-linux-gnu/__**gcc/x86_64-linux-gnu/4.5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but >> I feel like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >> >> >> wrote: >> >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually >> located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >> >> >>_**_wrote: >> >> >> > >> > > Ok then. Now I don't have enough experience with Eclipse so >> > > I apologize beforehand if you already know these/have >> tried them out. If >> > > not, hopefully they can be of help. I assume there >> should be a way in >> > > Eclipse to give it the link lib directory. In plain >> makefile that's just a >> > > simple step when linking. To get all the needed >> linklibs for petsc, you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, >> they are all located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with >> your makefile. I'm not >> > > sure about how you give them to Eclipse though. >> Hopefully this has been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> >> >> wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >> >> >>_**_wrote: >> >> >> > >> >> > >>> So do you want to be able to compile PETSc with >> Eclipse or just point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> >> >> wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI >> applications so I >> > >>>> am familiar with it, but I've never tried doing a >> non-Qt project in it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are >> makefiles that it >> > >>>> generates (which are for GNU make) but I think I can >> also manually create my >> > >>>> makefiles. After sleeping on it, it seems like this >> might be the best >> > >>>> option, unless I can figure out a way to configure >> eclipse to include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >> >> > >> >> >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do >> right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to >> Eclipse. It links nicely >> > >>>>> with PETSc(or any other library for that matter) >> and has excellent c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >> >>_**_wrote: >> >> >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc >> users manual about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses >> PETSc from Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge >> of how to do indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov >> >> > >> >> >> >>}. >> >> >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and >> then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and >> selected shared >> > >>>>>> library. After this point you can perform searchs >> in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to >> build an Eclipse >> > >>>>>> index for PETSc that can be used with their own >> code without compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create >> a symlink to the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so >> the new symlink is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project >> explorer, and choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the >> project explorer, and >> > >>>>>> choose "Exclude from build..." to make sure >> Eclipse does not try to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it >> right and include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >> Eclipse nicely generates >> > >>>>>> all my makefiles for me for my current project >> (which is written in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm >> not sure how to do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.van-wachem at imperial.ac.uk Wed Jul 27 19:54:09 2011 From: b.van-wachem at imperial.ac.uk (Berend van Wachem) Date: Thu, 28 Jul 2011 01:54:09 +0100 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> Message-ID: <4E30B331.5090709@imperial.ac.uk> Dear Matt, The error is definitely in the compiler stage: it cannot find that appropriate petsc.h include file. 1. Have you included petsc.h in the source file? 2. What do you have under -> Settings -> Includes ? Also, just to make sure, you can set the appropriate values for PETSC_DIR and PETSC_ARCH if you wish under -> Settings -> Environment Regards, Berend. On 07/28/2011 01:49 AM, Matt Bockman wrote: > Hi Berend, > > Here is the complete output of the build: > > http://pastebin.com/Es4ms4EF > > and by the 2nd to last line "collect2: ld returned 1 exit status" I > believe it is failing during the linking step. I'm not 100% sure though. > > Thanks for your quick response and help. Please advise, > > Matt > > On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem > > wrote: > > Dear Matt, > > Does it say this during the compiling? Or linking? > > If it says this during the compiling, it means that eclipse cannot > find the PETSc header files. So, it must be the setting of the > "Includes". You might want to "hard-code" the directory, just to > make sure. > > It is indeed not completely straightforward - eclipse has so many > options. But trust me - many of them you will really learn to > appreciate over time. > > Kind regards, > > Berend. > > > > On 07/28/2011 01:34 AM, Matt Bockman wrote: > > Thanks Berend for your thorough response, > > I have done what you have said but I still get the same error > regarding > "undefined references to PetscInitialize" etc. It's like I didn't > include the petscksp.h file, but it's there. I even tried > petsc.h to no > avail. > > I'm not sure what the compiler is referring to when it says > "Undefined > references to ...". What I think this is is in the assembly code > generated by the compiler, there is a PetscInitialize symbol > that isn't > found in the library. But I'm soooooooooo confused at this point > :(. How > did you guys all learn how to compile this? > > Matt > > On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem > > >> wrote: > > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where the PETSc > headers/libraries are to be found, so if you have a C project which > needs PETSc headers and libraries: > > To do this, click on your managed C project with the right sided > mouse button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side the various > setting options. > > Select Includes, and add the required PETSc paths. In my case I have > added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/____include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, > rt,gcc_s, pthread, X11 > (you can find these easily in > $PETSC_DIR/$PETSC_ARCH/____petscmachineinfo.h) > > The nice thing is that in eclipse you can easily switch between > Debug/Release code, traverse into the PETSc source code etc. It's > really a very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > > Thanks everyone for the help, > > I was able to compile a single example in Eclipse using the provided > makefile. I'm pretty new to makefiles so it's a LOT to digest. > I'm now > manually creating a makefile for my project in Eclipse (and I've set > Eclipse up to use a makefile that I create instead of automatically > generating one). Unfortunately this is a big pain but since I can't > figure out how to make Eclipse automatically include a few files > in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > > > > >>> wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the > only file you are compiling, you need to change the function name to > main. > 2) linking should be done after object files are created. A simple > g++ call would first compile the main file and then link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. > So wherever in Eclipse that you are seting the parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > > > > >>> wrote: > > I added the include directories from "make getincludedirs" and I > added the line from "make getlinklib". Eclipse creates a gcc > call as follows: > > /home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/bin/____mpicc > -I/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/__include > -I/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/include > -I/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/__include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/____Documents/Research/codes/____petsc/petsc-3.1-p8/linux-gnu-____c-debug/lib > -Wl,-rpath,/home/mdbockman/____Documents/Research/codes/____petsc/petsc-3.1-p8/linux-gnu-____c-debug/lib > -L/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/____Documents/Research/codes/____petsc/petsc-3.1-p8/linux-gnu-____c-debug/lib > -L/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/____gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but > I feel like I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > > > > >>> wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > > > > > >>>____wrote: > > > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > > > > >>> wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > > > > > >>>____wrote: > > > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > > > > >>> wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems like this > might be the best > > >>>> option, unless I can figure out a way to configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh > > > > > >> > > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > > > > > >>>____wrote: > > > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > > > > ____anl.gov > >>}. > > > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > > From mdbockma at ucsd.edu Wed Jul 27 20:04:49 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 18:04:49 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <4E30B331.5090709@imperial.ac.uk> References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> Message-ID: Hi Berend, Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> Directories : Include paths (-I) I have: /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include Under Properties -> C/C++ BUild -> Environment -> I have PETSC_ARCH = linux-gnu-c-debug PETSC_DIR = /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8 In my source file I have included petsc.h: #include "petsc.h" Thanks, Matt On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem < b.van-wachem at imperial.ac.uk> wrote: > Dear Matt, > > The error is definitely in the compiler stage: it cannot find that > appropriate petsc.h include file. > 1. Have you included petsc.h in the source file? > 2. What do you have under > -> Settings -> Includes > ? > > Also, just to make sure, you can set the appropriate values for PETSC_DIR > and PETSC_ARCH if you wish under > -> Settings -> Environment > > Regards, > > Berend. > > > > On 07/28/2011 01:49 AM, Matt Bockman wrote: > >> Hi Berend, >> >> Here is the complete output of the build: >> >> http//pastebin.com/Es4ms4EF >> >> and by the 2nd to last line "collect2: ld returned 1 exit status" I >> believe it is failing during the linking step. I'm not 100% sure though. >> >> Thanks for your quick response and help. Please advise, >> >> Matt >> >> On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem >> >> >> wrote: >> >> Dear Matt, >> >> Does it say this during the compiling? Or linking? >> >> If it says this during the compiling, it means that eclipse cannot >> find the PETSc header files. So, it must be the setting of the >> "Includes". You might want to "hard-code" the directory, just to >> make sure. >> >> It is indeed not completely straightforward - eclipse has so many >> options. But trust me - many of them you will really learn to >> appreciate over time. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/28/2011 01:34 AM, Matt Bockman wrote: >> >> Thanks Berend for your thorough response, >> >> I have done what you have said but I still get the same error >> regarding >> "undefined references to PetscInitialize" etc. It's like I didn't >> include the petscksp.h file, but it's there. I even tried >> petsc.h to no >> avail. >> >> I'm not sure what the compiler is referring to when it says >> "Undefined >> references to ...". What I think this is is in the assembly code >> generated by the compiler, there is a PetscInitialize symbol >> that isn't >> found in the library. But I'm soooooooooo confused at this point >> :(. How >> did you guys all learn how to compile this? >> >> Matt >> >> On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem >> > >> > >> > >>> >> wrote: >> >> Dear Matt, >> >> I use Eclipse and have eclipse make the makefiles. >> It is just a matter of indicating to eclipse where the PETSc >> headers/libraries are to be found, so if you have a C project which >> needs PETSc headers and libraries: >> >> To do this, click on your managed C project with the right sided >> mouse button, select >> >> Properties -> C/C++ Build -> Settings >> >> Then you get a new window with on the right hand side the various >> setting options. >> >> Select Includes, and add the required PETSc paths. In my case I >> have >> added >> ${PETSC_DIR}/include >> ${PETSC_DIR}/${PETSC_ARCH}/___**_include >> >> Then select "Libraries" under the header Linker >> and you should set the Library search path: >> ${PETSC_DIR}/${PETSC_ARCH}/lib >> >> and then the libraries, in my case: >> m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, >> rt,gcc_s, pthread, X11 >> (you can find these easily in >> $PETSC_DIR/$PETSC_ARCH/____**petscmachineinfo.h) >> >> The nice thing is that in eclipse you can easily switch between >> Debug/Release code, traverse into the PETSc source code etc. It's >> really a very productive tool with PETSc I've found. >> >> Let me know if you have any questions. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/27/2011 11:25 PM, Matt Bockman wrote: >> >> Thanks everyone for the help, >> >> I was able to compile a single example in Eclipse using the >> provided >> makefile. I'm pretty new to makefiles so it's a LOT to digest. >> I'm now >> manually creating a makefile for my project in Eclipse (and I've >> set >> Eclipse up to use a makefile that I create instead of automatically >> generating one). Unfortunately this is a big pain but since I can't >> figure out how to make Eclipse automatically include a few files >> in the >> makefile I don't really have any other choices :(. >> >> Thanks again, >> Matt >> >> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh >> >> > >> >> >>> wrote: >> >> There two problems(I think) in this code. >> >> 1) there is no main function in your source code. If this is the >> only file you are compiling, you need to change the function name >> to >> main. >> 2) linking should be done after object files are created. A simple >> g++ call would first compile the main file and then link the object >> to the petsc lib i.e >> >> g++ -c -I($PETSC_INCLUDE) main.cpp >> g++ -o main main.o $PETSC_LIBS >> >> alternatively, you could do it in a single line if you like >> >> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >> >> my point is you should link to petsc after compiling your own code. >> So wherever in Eclipse that you are seting the parameters, make >> sure >> the $PETSC_LIBS is in the linker option and not compiler. >> >> Mohammad >> >> >> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman >> >> > >> >> >>> wrote: >> >> I added the include directories from "make getincludedirs" and I >> added the line from "make getlinklib". Eclipse creates a gcc >> call as follows: >> >> /home/mdbockman/Documents/____**Research/codes/petsc/petsc-3._** >> ___1-p8/linux-gnu-c-debug/bin/**____mpicc >> -I/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/linux-gnu-c-debug/_**_include >> -I/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/include >> -I/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/linux-gnu-c-debug/_**_include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/___**_Documents/Research/codes/____** >> petsc/petsc-3.1-p8/linux-gnu-_**___c-debug/lib >> -Wl,-rpath,/home/mdbockman/___**_Documents/Research/codes/____** >> petsc/petsc-3.1-p8/linux-gnu-_**___c-debug/lib >> -L/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/linux-gnu-c-debug/**lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/___**_Documents/Research/codes/____** >> petsc/petsc-3.1-p8/linux-gnu-_**___c-debug/lib >> -L/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/linux-gnu-c-debug/**lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/__**__Research/codes/petsc/petsc-** >> 3.____1-p8/linux-gnu-c-debug/**lib >> -L/usr/lib/x86_64-linux-gnu/__**__gcc/x86_64-linux-gnu/4.5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but >> I feel like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >> >> > >> >> >>> wrote: >> >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually >> located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >> >> > >> >> >>>** >> ____wrote: >> >> >> > >> > > Ok then. Now I don't have enough experience with Eclipse so >> > > I apologize beforehand if you already know these/have >> tried them out. If >> > > not, hopefully they can be of help. I assume there >> should be a way in >> > > Eclipse to give it the link lib directory. In plain >> makefile that's just a >> > > simple step when linking. To get all the needed >> linklibs for petsc, you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, >> they are all located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with >> your makefile. I'm not >> > > sure about how you give them to Eclipse though. >> Hopefully this has been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> >> > >> >> >>> wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >> >> > >> >> >>>** >> ____wrote: >> >> >> > >> >> > >>> So do you want to be able to compile PETSc with >> Eclipse or just point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> >> > >> >> >>> wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI >> applications so I >> > >>>> am familiar with it, but I've never tried doing a >> non-Qt project in it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are >> makefiles that it >> > >>>> generates (which are for GNU make) but I think I can >> also manually create my >> > >>>> makefiles. After sleeping on it, it seems like this >> might be the best >> > >>>> option, unless I can figure out a way to configure >> eclipse to include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do >> right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to >> Eclipse. It links nicely >> > >>>>> with PETSc(or any other library for that matter) >> and has excellent c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >> > >> >> >>>** >> ____wrote: >> >> >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc >> users manual about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses >> PETSc from Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge >> of how to do indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov >> >> > >> >> >> >> >> > ____anl.gov >> >> > >> >>>}. >> >> >> >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and >> then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and >> selected shared >> > >>>>>> library. After this point you can perform searchs >> in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to >> build an Eclipse >> > >>>>>> index for PETSc that can be used with their own >> code without compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create >> a symlink to the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so >> the new symlink is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project >> explorer, and choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the >> project explorer, and >> > >>>>>> choose "Exclude from build..." to make sure >> Eclipse does not try to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it >> right and include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >> Eclipse nicely generates >> > >>>>>> all my makefiles for me for my current project >> (which is written in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm >> not sure how to do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.van-wachem at imperial.ac.uk Wed Jul 27 20:11:58 2011 From: b.van-wachem at imperial.ac.uk (Berend van Wachem) Date: Thu, 28 Jul 2011 02:11:58 +0100 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> Message-ID: <4E30B75E.2080805@imperial.ac.uk> Dear Matt, What do you have under Properties->C/C++ Build -> Settings -> GCC Linker Libraries for both "Libraries" and "Library Search Path"? Regards, Berend. On 07/28/2011 02:04 AM, Matt Bockman wrote: > Hi Berend, > > Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> > Directories : Include paths (-I) I have: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > > Under Properties -> C/C++ BUild -> Environment -> I have > > PETSC_ARCH = linux-gnu-c-debug > PETSC_DIR = /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8 > > In my source file I have included petsc.h: > > #include "petsc.h" > > Thanks, > Matt > > On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem > > wrote: > > Dear Matt, > > The error is definitely in the compiler stage: it cannot find that > appropriate petsc.h include file. > 1. Have you included petsc.h in the source file? > 2. What do you have under > -> Settings -> Includes > ? > > Also, just to make sure, you can set the appropriate values for > PETSC_DIR and PETSC_ARCH if you wish under > -> Settings -> Environment > > Regards, > > Berend. > > > > On 07/28/2011 01:49 AM, Matt Bockman wrote: > > Hi Berend, > > Here is the complete output of the build: > > http//pastebin.com/Es4ms4EF > > and by the 2nd to last line "collect2: ld returned 1 exit status" I > believe it is failing during the linking step. I'm not 100% sure > though. > > Thanks for your quick response and help. Please advise, > > Matt > > On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem > > >> wrote: > > Dear Matt, > > Does it say this during the compiling? Or linking? > > If it says this during the compiling, it means that eclipse cannot > find the PETSc header files. So, it must be the setting of the > "Includes". You might want to "hard-code" the directory, just to > make sure. > > It is indeed not completely straightforward - eclipse has so many > options. But trust me - many of them you will really learn to > appreciate over time. > > Kind regards, > > Berend. > > > > On 07/28/2011 01:34 AM, Matt Bockman wrote: > > Thanks Berend for your thorough response, > > I have done what you have said but I still get the same error > regarding > "undefined references to PetscInitialize" etc. It's like I didn't > include the petscksp.h file, but it's there. I even tried > petsc.h to no > avail. > > I'm not sure what the compiler is referring to when it says > "Undefined > references to ...". What I think this is is in the assembly code > generated by the compiler, there is a PetscInitialize symbol > that isn't > found in the library. But I'm soooooooooo confused at this point > :(. How > did you guys all learn how to compile this? > > Matt > > On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem > > > > ____ac.uk > >>> wrote: > > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where the PETSc > headers/libraries are to be found, so if you have a C project which > needs PETSc headers and libraries: > > To do this, click on your managed C project with the right sided > mouse button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side the various > setting options. > > Select Includes, and add the required PETSc paths. In my case I have > added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/______include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, > rt,gcc_s, pthread, X11 > (you can find these easily in > $PETSC_DIR/$PETSC_ARCH/______petscmachineinfo.h) > > The nice thing is that in eclipse you can easily switch between > Debug/Release code, traverse into the PETSc source code etc. It's > really a very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > > Thanks everyone for the help, > > I was able to compile a single example in Eclipse using the provided > makefile. I'm pretty new to makefiles so it's a LOT to digest. > I'm now > manually creating a makefile for my project in Eclipse (and I've set > Eclipse up to use a makefile that I create instead of automatically > generating one). Unfortunately this is a big pain but since I can't > figure out how to make Eclipse automatically include a few files > in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > > > > >> > > > > > >>>> wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the > only file you are compiling, you need to change the function name to > main. > 2) linking should be done after object files are created. A simple > g++ call would first compile the main file and then link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. > So wherever in Eclipse that you are seting the parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > > > > >> > > > > > >>>> wrote: > > I added the include directories from "make getincludedirs" and I > added the line from "make getlinklib". Eclipse creates a gcc > call as follows: > > /home/mdbockman/Documents/______Research/codes/petsc/petsc-3.______1-p8/linux-gnu-c-debug/bin/______mpicc > -I/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/linux-gnu-c-debug/____include > -I/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/include > -I/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/linux-gnu-c-debug/____include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/______Documents/Research/codes/______petsc/petsc-3.1-p8/linux-gnu-______c-debug/lib > -Wl,-rpath,/home/mdbockman/______Documents/Research/codes/______petsc/petsc-3.1-p8/linux-gnu-______c-debug/lib > -L/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/linux-gnu-c-debug/__lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/______Documents/Research/codes/______petsc/petsc-3.1-p8/linux-gnu-______c-debug/lib > -L/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/linux-gnu-c-debug/__lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/______Research/codes/petsc/petsc-__3.____1-p8/linux-gnu-c-debug/__lib > -L/usr/lib/x86_64-linux-gnu/______gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but > I feel like I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > > > > >> > > > > > >>>> wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>>>______wrote: > > > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > > > > >> > > > > > >>>> wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>>>______wrote: > > > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > > > > >> > > > > > >>>> wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems like this > might be the best > > >>>> option, unless I can figure out a way to configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>> > > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > > > > > >> > > > > > >>>>______wrote: > > > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > > > > ____anl.gov > >> > . > .__>____anl.gov > > > ____anl.gov > >>>}. > > > > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > > > From mdbockma at ucsd.edu Wed Jul 27 20:20:20 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 18:20:20 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <4E30B75E.2080805@imperial.ac.uk> References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> <4E30B75E.2080805@imperial.ac.uk> Message-ID: Hi Berend, Under Library search path I have: /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib For libraries I have: petsc X11 flapack fblas nsl rt m dl mpich pthread rt gcc_s mpichf90 gfortran m m dl mpich rt gcc_s dl I used the exact order as I found in petscmachineinfo.h file (see here: http://pastebin.com/cqzNgjJe) Thank you very much for your help Berend, Matt On Wed, Jul 27, 2011 at 6:11 PM, Berend van Wachem < b.van-wachem at imperial.ac.uk> wrote: > Dear Matt, > > > What do you have under > Properties->C/C++ Build -> Settings -> GCC Linker Libraries for both > "Libraries" and "Library Search Path"? > > Regards, > Berend. > > > > On 07/28/2011 02:04 AM, Matt Bockman wrote: > >> Hi Berend, >> >> Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> >> Directories : Include paths (-I) I have: >> >> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.**1-p8/include >> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/include >> >> Under Properties -> C/C++ BUild -> Environment -> I have >> >> PETSC_ARCH = linux-gnu-c-debug >> PETSC_DIR = /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8 >> >> In my source file I have included petsc.h: >> >> #include "petsc.h" >> >> Thanks, >> Matt >> >> On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem >> >> >> wrote: >> >> Dear Matt, >> >> The error is definitely in the compiler stage: it cannot find that >> appropriate petsc.h include file. >> 1. Have you included petsc.h in the source file? >> 2. What do you have under >> -> Settings -> Includes >> ? >> >> Also, just to make sure, you can set the appropriate values for >> PETSC_DIR and PETSC_ARCH if you wish under >> -> Settings -> Environment >> >> Regards, >> >> Berend. >> >> >> >> On 07/28/2011 01:49 AM, Matt Bockman wrote: >> >> Hi Berend, >> >> Here is the complete output of the build: >> >> http//pastebin.com/Es4ms4EF >> >> >> and by the 2nd to last line "collect2: ld returned 1 exit status" I >> believe it is failing during the linking step. I'm not 100% sure >> though. >> >> Thanks for your quick response and help. Please advise, >> >> Matt >> >> On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem >> > >> > >> > >>> >> wrote: >> >> Dear Matt, >> >> Does it say this during the compiling? Or linking? >> >> If it says this during the compiling, it means that eclipse cannot >> find the PETSc header files. So, it must be the setting of the >> "Includes". You might want to "hard-code" the directory, just to >> make sure. >> >> It is indeed not completely straightforward - eclipse has so many >> options. But trust me - many of them you will really learn to >> appreciate over time. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/28/2011 01:34 AM, Matt Bockman wrote: >> >> Thanks Berend for your thorough response, >> >> I have done what you have said but I still get the same error >> regarding >> "undefined references to PetscInitialize" etc. It's like I didn't >> include the petscksp.h file, but it's there. I even tried >> petsc.h to no >> avail. >> >> I'm not sure what the compiler is referring to when it says >> "Undefined >> references to ...". What I think this is is in the assembly code >> generated by the compiler, there is a PetscInitialize symbol >> that isn't >> found in the library. But I'm soooooooooo confused at this point >> :(. How >> did you guys all learn how to compile this? >> >> Matt >> >> On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem >> >> > >> > >> >> >> > ____ac.uk >> >> > >>>> >> wrote: >> >> Dear Matt, >> >> I use Eclipse and have eclipse make the makefiles. >> It is just a matter of indicating to eclipse where the PETSc >> headers/libraries are to be found, so if you have a C project which >> needs PETSc headers and libraries: >> >> To do this, click on your managed C project with the right sided >> mouse button, select >> >> Properties -> C/C++ Build -> Settings >> >> Then you get a new window with on the right hand side the various >> setting options. >> >> Select Includes, and add the required PETSc paths. In my case I >> have >> added >> ${PETSC_DIR}/include >> ${PETSC_DIR}/${PETSC_ARCH}/___**___include >> >> Then select "Libraries" under the header Linker >> and you should set the Library search path: >> ${PETSC_DIR}/${PETSC_ARCH}/lib >> >> and then the libraries, in my case: >> m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, >> rt,gcc_s, pthread, X11 >> (you can find these easily in >> $PETSC_DIR/$PETSC_ARCH/______**petscmachineinfo.h) >> >> The nice thing is that in eclipse you can easily switch between >> Debug/Release code, traverse into the PETSc source code etc. It's >> really a very productive tool with PETSc I've found. >> >> Let me know if you have any questions. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/27/2011 11:25 PM, Matt Bockman wrote: >> >> Thanks everyone for the help, >> >> I was able to compile a single example in Eclipse using the >> provided >> makefile. I'm pretty new to makefiles so it's a LOT to digest. >> I'm now >> manually creating a makefile for my project in Eclipse (and I've >> set >> Eclipse up to use a makefile that I create instead of automatically >> generating one). Unfortunately this is a big pain but since I can't >> figure out how to make Eclipse automatically include a few files >> in the >> makefile I don't really have any other choices :(. >> >> Thanks again, >> Matt >> >> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>>**> >> wrote: >> >> There two problems(I think) in this code. >> >> 1) there is no main function in your source code. If this is the >> only file you are compiling, you need to change the function name >> to >> main. >> 2) linking should be done after object files are created. A simple >> g++ call would first compile the main file and then link the object >> to the petsc lib i.e >> >> g++ -c -I($PETSC_INCLUDE) main.cpp >> g++ -o main main.o $PETSC_LIBS >> >> alternatively, you could do it in a single line if you like >> >> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >> >> my point is you should link to petsc after compiling your own code. >> So wherever in Eclipse that you are seting the parameters, make >> sure >> the $PETSC_LIBS is in the linker option and not compiler. >> >> Mohammad >> >> >> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>>> wrote: >> >> I added the include directories from "make getincludedirs" and I >> added the line from "make getlinklib". Eclipse creates a gcc >> call as follows: >> >> /home/mdbockman/Documents/____**__Research/codes/petsc/petsc-** >> 3.______1-p8/linux-gnu-c-**debug/bin/______mpicc >> -I/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/linux-gnu-**c-debug/____include >> -I/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/include >> -I/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/linux-gnu-**c-debug/____include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/___**___Documents/Research/codes/__** >> ____petsc/petsc-3.1-p8/linux-**gnu-______c-debug/lib >> -Wl,-rpath,/home/mdbockman/___**___Documents/Research/codes/__** >> ____petsc/petsc-3.1-p8/linux-**gnu-______c-debug/lib >> -L/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/linux-gnu-**c-debug/__lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/___**___Documents/Research/codes/__** >> ____petsc/petsc-3.1-p8/linux-**gnu-______c-debug/lib >> -L/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/linux-gnu-**c-debug/__lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/__**____Research/codes/petsc/** >> petsc-__3.____1-p8/linux-gnu-**c-debug/__lib >> -L/usr/lib/x86_64-linux-gnu/__**____gcc/x86_64-linux-gnu/4.5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but >> I feel like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >> >> > >> >> >> >> >> > >> >> >>>> wrote: >> >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually >> located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> > >>>**>______wrote: >> >> >> > >> > > Ok then. Now I don't have enough experience with Eclipse so >> > > I apologize beforehand if you already know these/have >> tried them out. If >> > > not, hopefully they can be of help. I assume there >> should be a way in >> > > Eclipse to give it the link lib directory. In plain >> makefile that's just a >> > > simple step when linking. To get all the needed >> linklibs for petsc, you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, >> they are all located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with >> your makefile. I'm not >> > > sure about how you give them to Eclipse though. >> Hopefully this has been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>>> wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> > >>>**>______wrote: >> >> >> > >> >> > >>> So do you want to be able to compile PETSc with >> Eclipse or just point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>>> wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI >> applications so I >> > >>>> am familiar with it, but I've never tried doing a >> non-Qt project in it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are >> makefiles that it >> > >>>> generates (which are for GNU make) but I think I can >> also manually create my >> > >>>> makefiles. After sleeping on it, it seems like this >> might be the best >> > >>>> option, unless I can figure out a way to configure >> eclipse to include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>> >> >> >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do >> right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to >> Eclipse. It links nicely >> > >>>>> with PETSc(or any other library for that matter) >> and has excellent c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >> > >> >> >> >> >> > >> >> > >>>**>______wrote: >> >> >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc >> users manual about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses >> PETSc from Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge >> of how to do indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov >> >> > >> >> >> >> >> > ____anl.gov >> > >> >>> >> **. >> > >> >.__>____anl.gov >> >> >> >> > >> .>____anl.gov >> >> >> >> >>>>}. >> >> >> >> >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and >> then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and >> selected shared >> > >>>>>> library. After this point you can perform searchs >> in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to >> build an Eclipse >> > >>>>>> index for PETSc that can be used with their own >> code without compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create >> a symlink to the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so >> the new symlink is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project >> explorer, and choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the >> project explorer, and >> > >>>>>> choose "Exclude from build..." to make sure >> Eclipse does not try to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it >> right and include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >> Eclipse nicely generates >> > >>>>>> all my makefiles for me for my current project >> (which is written in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm >> not sure how to do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From b.van-wachem at imperial.ac.uk Wed Jul 27 20:28:52 2011 From: b.van-wachem at imperial.ac.uk (Berend van Wachem) Date: Thu, 28 Jul 2011 02:28:52 +0100 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> <4E30B75E.2080805@imperial.ac.uk> Message-ID: <4E30BB54.6040106@imperial.ac.uk> Dear Matt, I can't directly see anything wrong with your settings. Have you tried compiling your example outside of eclipse with the PETSc makefile system? The previous pastebin output you emailed me does suggest that the error is during the compile stage: it cannot find the correct headers describing the implementations of PetscInitialize etc. Have you tried using the mpicc outside of eclipse directly on the directory? /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc -I"/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include" -I"/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include" -O0 -g3 -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP -MF"ex1.d" -MT"ex1.d" -o"ex1.o" "../ex1.c" within the eclipse directory? This way you can more easily play around with it to see what is the problem. I'm sorry I can't be of more help. Regards, Berend. On 07/28/2011 02:20 AM, Matt Bockman wrote: > Hi Berend, > > Under Library search path I have: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > > For libraries I have: > > petsc > X11 > flapack > fblas > nsl > rt > m > dl > mpich > pthread > rt > gcc_s > mpichf90 > gfortran > m > m > dl > mpich > rt > gcc_s > dl > > I used the exact order as I found in petscmachineinfo.h file (see here: > http://pastebin.com/cqzNgjJe) > > Thank you very much for your help Berend, > Matt > > On Wed, Jul 27, 2011 at 6:11 PM, Berend van Wachem > > wrote: > > Dear Matt, > > > What do you have under > Properties->C/C++ Build -> Settings -> GCC Linker Libraries for both > "Libraries" and "Library Search Path"? > > Regards, > Berend. > > > > On 07/28/2011 02:04 AM, Matt Bockman wrote: > > Hi Berend, > > Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> > Directories : Include paths (-I) I have: > > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/include > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/include > > Under Properties -> C/C++ BUild -> Environment -> I have > > PETSC_ARCH = linux-gnu-c-debug > PETSC_DIR = > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8 > > In my source file I have included petsc.h: > > #include "petsc.h" > > Thanks, > Matt > > On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem > > >> wrote: > > Dear Matt, > > The error is definitely in the compiler stage: it cannot find that > appropriate petsc.h include file. > 1. Have you included petsc.h in the source file? > 2. What do you have under > -> Settings -> Includes > ? > > Also, just to make sure, you can set the appropriate values for > PETSC_DIR and PETSC_ARCH if you wish under > -> Settings -> Environment > > Regards, > > Berend. > > > > On 07/28/2011 01:49 AM, Matt Bockman wrote: > > Hi Berend, > > Here is the complete output of the build: > > http//pastebin.com/Es4ms4EF > > > > and by the 2nd to last line "collect2: ld returned 1 exit status" I > believe it is failing during the linking step. I'm not 100% sure > though. > > Thanks for your quick response and help. Please advise, > > Matt > > On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem > > > > ____ac.uk > >>> wrote: > > Dear Matt, > > Does it say this during the compiling? Or linking? > > If it says this during the compiling, it means that eclipse cannot > find the PETSc header files. So, it must be the setting of the > "Includes". You might want to "hard-code" the directory, just to > make sure. > > It is indeed not completely straightforward - eclipse has so many > options. But trust me - many of them you will really learn to > appreciate over time. > > Kind regards, > > Berend. > > > > On 07/28/2011 01:34 AM, Matt Bockman wrote: > > Thanks Berend for your thorough response, > > I have done what you have said but I still get the same error > regarding > "undefined references to PetscInitialize" etc. It's like I didn't > include the petscksp.h file, but it's there. I even tried > petsc.h to no > avail. > > I'm not sure what the compiler is referring to when it says > "Undefined > references to ...". What I think this is is in the assembly code > generated by the compiler, there is a PetscInitialize symbol > that isn't > found in the library. But I'm soooooooooo confused at this point > :(. How > did you guys all learn how to compile this? > > Matt > > On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem > > > > ____ac.uk > >> > . > .__>____ac.uk > > > ____ac.uk > >>>> wrote: > > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where the PETSc > headers/libraries are to be found, so if you have a C project which > needs PETSc headers and libraries: > > To do this, click on your managed C project with the right sided > mouse button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side the various > setting options. > > Select Includes, and add the required PETSc paths. In my case I have > added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/________include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, > rt,gcc_s, pthread, X11 > (you can find these easily in > $PETSC_DIR/$PETSC_ARCH/________petscmachineinfo.h) > > The nice thing is that in eclipse you can easily switch between > Debug/Release code, traverse into the PETSc source code etc. It's > really a very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > > Thanks everyone for the help, > > I was able to compile a single example in Eclipse using the provided > makefile. I'm pretty new to makefiles so it's a LOT to digest. > I'm now > manually creating a makefile for my project in Eclipse (and I've set > Eclipse up to use a makefile that I create instead of automatically > generating one). Unfortunately this is a big pain but since I can't > figure out how to make Eclipse automatically include a few files > in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>__> wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If this is the > only file you are compiling, you need to change the function name to > main. > 2) linking should be done after object files are created. A simple > g++ call would first compile the main file and then link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling your own code. > So wherever in Eclipse that you are seting the parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>> wrote: > > I added the include directories from "make getincludedirs" and I > added the line from "make getlinklib". Eclipse creates a gcc > call as follows: > > /home/mdbockman/Documents/________Research/codes/petsc/petsc-__3.______1-p8/linux-gnu-c-__debug/bin/______mpicc > -I/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/linux-gnu-__c-debug/____include > -I/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/include > -I/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/linux-gnu-__c-debug/____include > -O0 -g3 -pg -p -Wall > -Wl,-rpath,/home/mdbockman/________Documents/Research/codes/________petsc/petsc-3.1-p8/linux-__gnu-______c-debug/lib > -Wl,-rpath,/home/mdbockman/________Documents/Research/codes/________petsc/petsc-3.1-p8/linux-__gnu-______c-debug/lib > -L/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/linux-gnu-__c-debug/__lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/________Documents/Research/codes/________petsc/petsc-3.1-p8/linux-__gnu-______c-debug/lib > -L/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/linux-gnu-__c-debug/__lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/________Research/codes/petsc/__petsc-__3.____1-p8/linux-gnu-__c-debug/__lib > -L/usr/lib/x86_64-linux-gnu/________gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing wrong but > I feel like I'm getting closer and closer to the solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>> wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>__>______wrote: > > > > > > > Ok then. Now I don't have enough experience with Eclipse so > > > I apologize beforehand if you already know these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>> wrote: > > > > > >> Just pointing it to the library would be sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>__>______wrote: > > > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>> wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems like this > might be the best > > >>>> option, unless I can figure out a way to configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>__>______wrote: > > > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > > > > ____anl.gov > >> > . > .__>____anl.gov > > ____anl.gov > >>> > > >__. > > >.__>____anl.gov > > > > > . > .>____anl.gov > > > ____mcs.anl.gov > >>>>}. > > > > > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > > > > From mdbockma at ucsd.edu Wed Jul 27 21:21:27 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 19:21:27 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: <4E30BB54.6040106@imperial.ac.uk> References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> <4E30B75E.2080805@imperial.ac.uk> <4E30BB54.6040106@imperial.ac.uk> Message-ID: Hi Berend, First of all, you have been extremely helpful. Second, I was FINALLY able to compile the example through Eclipse. Sadly, it's the biggest breakthrough I've had in the last 2 weeks of my research :(. Here's what I did: I tried running what Eclipse was running from a terminal: Eclipse: /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -O0 -g3 -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP -MF"ex1.d" -MT"ex1.d" -o"ex1.o" "../ex1.c" That did not work. So I ran make ex1 in the original ex1 tutorial directory and saw what the makefile was doing there: /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc -o ex1.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -g3 -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex1.c /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -g3 -o ex1 ex1.o -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -lpetsc -lX11 -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -lflapack -lfblas -lnsl -lrt -lm -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl So apparently it was making an object file then linking. I then compared the one from Eclipse and found that I was missing -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve. Why I need this I do not know (ideas?). But, after I found that I added it to my Include Directories and it compiled! Thanks Berend and others, I appreciate your persistence and aid. Matt On Wed, Jul 27, 2011 at 6:28 PM, Berend van Wachem < b.van-wachem at imperial.ac.uk> wrote: > Dear Matt, > > I can't directly see anything wrong with your settings. > > Have you tried compiling your example outside of eclipse with the PETSc > makefile system? > > The previous pastebin output you emailed me does suggest that the error is > during the compile stage: it cannot find the correct headers describing the > implementations of PetscInitialize etc. > > Have you tried using the mpicc outside of eclipse directly on the > directory? > > /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** > 1-p8/linux-gnu-c-debug/bin/**mpicc -I"/home/mdbockman/Documents/** > Research/codes/petsc/petsc-3.**1-p8/include" -I"/home/mdbockman/Documents/ > **Research/codes/petsc/petsc-3.**1-p8/linux-gnu-c-debug/**include" -O0 -g3 > -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP -MF"ex1.d" -MT"ex1.d" > -o"ex1.o" "../ex1.c" > > within the eclipse directory? This way you can more easily play around with > it to see what is the problem. > > I'm sorry I can't be of more help. > > Regards, > Berend. > > > > On 07/28/2011 02:20 AM, Matt Bockman wrote: > >> Hi Berend, >> >> Under Library search path I have: >> >> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/lib >> >> For libraries I have: >> >> petsc >> X11 >> flapack >> fblas >> nsl >> rt >> m >> dl >> mpich >> pthread >> rt >> gcc_s >> mpichf90 >> gfortran >> m >> m >> dl >> mpich >> rt >> gcc_s >> dl >> >> I used the exact order as I found in petscmachineinfo.h file (see here: >> http://pastebin.com/cqzNgjJe) >> >> Thank you very much for your help Berend, >> Matt >> >> On Wed, Jul 27, 2011 at 6:11 PM, Berend van Wachem >> >> >> wrote: >> >> Dear Matt, >> >> >> What do you have under >> Properties->C/C++ Build -> Settings -> GCC Linker Libraries for both >> "Libraries" and "Library Search Path"? >> >> Regards, >> Berend. >> >> >> >> On 07/28/2011 02:04 AM, Matt Bockman wrote: >> >> Hi Berend, >> >> Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> >> Directories : Include paths (-I) I have: >> >> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/include >> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8/linux-gnu-c-debug/**include >> >> Under Properties -> C/C++ BUild -> Environment -> I have >> >> PETSC_ARCH = linux-gnu-c-debug >> PETSC_DIR = >> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >> _1-p8 >> >> In my source file I have included petsc.h: >> >> #include "petsc.h" >> >> Thanks, >> Matt >> >> On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem >> > >> > >> > >>> >> wrote: >> >> Dear Matt, >> >> The error is definitely in the compiler stage: it cannot find that >> appropriate petsc.h include file. >> 1. Have you included petsc.h in the source file? >> 2. What do you have under >> -> Settings -> Includes >> ? >> >> Also, just to make sure, you can set the appropriate values for >> PETSC_DIR and PETSC_ARCH if you wish under >> -> Settings -> Environment >> >> Regards, >> >> Berend. >> >> >> >> On 07/28/2011 01:49 AM, Matt Bockman wrote: >> >> Hi Berend, >> >> Here is the complete output of the build: >> >> http//pastebin.com/Es4ms4EF >> >> >> >> and by the 2nd to last line "collect2: ld returned 1 exit status" I >> believe it is failing during the linking step. I'm not 100% sure >> though. >> >> Thanks for your quick response and help. Please advise, >> >> Matt >> >> On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem >> >> > >> > >> >> >> > ____ac.uk >> > >>>> >> wrote: >> >> Dear Matt, >> >> Does it say this during the compiling? Or linking? >> >> If it says this during the compiling, it means that eclipse cannot >> find the PETSc header files. So, it must be the setting of the >> "Includes". You might want to "hard-code" the directory, just to >> make sure. >> >> It is indeed not completely straightforward - eclipse has so many >> options. But trust me - many of them you will really learn to >> appreciate over time. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/28/2011 01:34 AM, Matt Bockman wrote: >> >> Thanks Berend for your thorough response, >> >> I have done what you have said but I still get the same error >> regarding >> "undefined references to PetscInitialize" etc. It's like I didn't >> include the petscksp.h file, but it's there. I even tried >> petsc.h to no >> avail. >> >> I'm not sure what the compiler is referring to when it says >> "Undefined >> references to ...". What I think this is is in the assembly code >> generated by the compiler, there is a PetscInitialize symbol >> that isn't >> found in the library. But I'm soooooooooo confused at this point >> :(. How >> did you guys all learn how to compile this? >> >> Matt >> >> On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem >> > >> > >> > >> >> >> > ____ac.uk >> > >> >>> >> **. >> > **.__>____ac.uk >> >> >> > ____ac.uk >> > >>>>> >> wrote: >> >> Dear Matt, >> >> I use Eclipse and have eclipse make the makefiles. >> It is just a matter of indicating to eclipse where the PETSc >> headers/libraries are to be found, so if you have a C project which >> needs PETSc headers and libraries: >> >> To do this, click on your managed C project with the right sided >> mouse button, select >> >> Properties -> C/C++ Build -> Settings >> >> Then you get a new window with on the right hand side the various >> setting options. >> >> Select Includes, and add the required PETSc paths. In my case I >> have >> added >> ${PETSC_DIR}/include >> ${PETSC_DIR}/${PETSC_ARCH}/___**_____include >> >> Then select "Libraries" under the header Linker >> and you should set the Library search path: >> ${PETSC_DIR}/${PETSC_ARCH}/lib >> >> and then the libraries, in my case: >> m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, >> rt,gcc_s, pthread, X11 >> (you can find these easily in >> $PETSC_DIR/$PETSC_ARCH/_______**_petscmachineinfo.h) >> >> The nice thing is that in eclipse you can easily switch between >> Debug/Release code, traverse into the PETSc source code etc. It's >> really a very productive tool with PETSc I've found. >> >> Let me know if you have any questions. >> >> Kind regards, >> >> Berend. >> >> >> >> On 07/27/2011 11:25 PM, Matt Bockman wrote: >> >> Thanks everyone for the help, >> >> I was able to compile a single example in Eclipse using the >> provided >> makefile. I'm pretty new to makefiles so it's a LOT to digest. >> I'm now >> manually creating a makefile for my project in Eclipse (and I've >> set >> Eclipse up to use a makefile that I create instead of automatically >> generating one). Unfortunately this is a big pain but since I can't >> figure out how to make Eclipse automatically include a few files >> in the >> makefile I don't really have any other choices :(. >> >> Thanks again, >> Matt >> >> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>**>__> >> wrote: >> >> There two problems(I think) in this code. >> >> 1) there is no main function in your source code. If this is the >> only file you are compiling, you need to change the function name >> to >> main. >> 2) linking should be done after object files are created. A simple >> g++ call would first compile the main file and then link the object >> to the petsc lib i.e >> >> g++ -c -I($PETSC_INCLUDE) main.cpp >> g++ -o main main.o $PETSC_LIBS >> >> alternatively, you could do it in a single line if you like >> >> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >> >> my point is you should link to petsc after compiling your own code. >> So wherever in Eclipse that you are seting the parameters, make >> sure >> the $PETSC_LIBS is in the linker option and not compiler. >> >> Mohammad >> >> >> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>>**> wrote: >> >> I added the include directories from "make getincludedirs" and I >> added the line from "make getlinklib". Eclipse creates a gcc >> call as follows: >> >> /home/mdbockman/Documents/____**____Research/codes/petsc/** >> petsc-__3.______1-p8/linux-**gnu-c-__debug/bin/______mpicc >> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/linux-gnu-_**_c-debug/____include >> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/include >> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/linux-gnu-_**_c-debug/____include >> -O0 -g3 -pg -p -Wall >> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >> -lpetsc -lX11 >> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >> -lflapack -lfblas -lnsl -lrt -lm >> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >> -L/usr/lib/x86_64-linux-gnu/__**______gcc/x86_64-linux-gnu/4.**5.2 >> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >> "../SparseMatrixPetsc.c >> >> And when it is compiled I get the following: >> >> http://pastebin.com/CbRzYcZj >> >> The source file which is being compiled is: >> >> http://pastebin.com/Q85hXvnS >> >> Please have a look. I'm not quite sure what I'm doing wrong but >> I feel like I'm getting closer and closer to the solution. >> >> Matt >> >> >> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>>**> wrote: >> >> use: >> make getincludedirs >> >> Satish >> >> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >> >> > I applogize for the mistake; Include files are actually >> located >> > in $PETSC_DIR/include >> > >> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> > >>>**>__>______wrote: >> >> >> > >> > > Ok then. Now I don't have enough experience with Eclipse so >> > > I apologize beforehand if you already know these/have >> tried them out. If >> > > not, hopefully they can be of help. I assume there >> should be a way in >> > > Eclipse to give it the link lib directory. In plain >> makefile that's just a >> > > simple step when linking. To get all the needed >> linklibs for petsc, you can >> > > do >> > > >> > > make getlinklibs >> > > >> > > in the $PETSC_DIR. As for the needed include files, >> they are all located >> > > in >> > > >> > > $PETSC_DIR/$PETSC_ARCH/include >> > > >> > > Again, its easy to use these directories along with >> your makefile. I'm not >> > > sure about how you give them to Eclipse though. >> Hopefully this has been >> > > helpful. >> > > >> > > Best, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>>**> wrote: >> > > >> > >> Just pointing it to the library would be sufficient. >> > >> >> > >> Matt >> > >> >> > >> >> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> > >>>**>__>______wrote: >> >> >> > >> >> > >>> So do you want to be able to compile PETSc with >> Eclipse or just point it >> > >>> to the library to use in your own applications? >> > >>> >> > >>> >> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>>**> wrote: >> > >>> >> > >>>> Thanks Mohammad, >> > >>>> >> > >>>> I'll give that a shot. I use Qt Creator for some GUI >> applications so I >> > >>>> am familiar with it, but I've never tried doing a >> non-Qt project in it. I'd >> > >>>> really like to get Eclipse to work. >> > >>>> >> > >>>> Regarding the makefiles for eclipse. There are >> makefiles that it >> > >>>> generates (which are for GNU make) but I think I can >> also manually create my >> > >>>> makefiles. After sleeping on it, it seems like this >> might be the best >> > >>>> option, unless I can figure out a way to configure >> eclipse to include the >> > >>>> conf/variables and conf/rules files in the makefile. >> > >>>> >> > >>>> Matt >> > >>>> >> > >>>> >> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> >>>**> >> >> >> > >>>> > wrote: >> > >>>> >> > >>>>> Although this is sort of orthogonal to what you do >> right now, >> > >>>>> I recommend Qt Creator as an alternative IDE to >> Eclipse. It links nicely >> > >>>>> with PETSc(or any other library for that matter) >> and has excellent c/c++ >> > >>>>> support. >> > >>>>> >> > >>>>> Mohammad >> > >>>>> >> > >>>>> >> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >> >> > >> >> >> >> >> > >> >> >>> >> >> > >> >> >> >> >> > >> >> > >>>**>__>______wrote: >> >> >> > >>>>> >> > >>>>>> >> > >>>>>> There is a tiny bit of information in the PETSc >> users manual about >> > >>>>>> Eclipse: >> > >>>>>> >> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >> > >>>>>> >> > >>>>>> If you are interested in developing code that uses >> PETSc from Eclipse >> > >>>>>> or developing PETSc in Eclipse and have knowledge >> of how to do indexing and >> > >>>>>> build libraries in Eclipse please contact us at \ >> > >>>>>> trl{petsc-dev at mcs.anl.gov >> >> > >> >> >> >> >> > ____anl.gov >> > >> >>> >> **. >> > >> >.__>____anl.gov >> >> > >> .>____anl.gov >> >> >> >>>> >> >> >> >>__. >> >> >> > >> > >> >>.__>____anl.gov >> >> >> >> >> >> >. >> > >> >.>____anl.gov >> >> >> > ____mcs.anl.gov < >> http://mcs.anl.gov> >> >> > >> >>>>>}. >> >> >> >> >> >> > >>>>>> >> > >>>>>> To make PETSc an Eclipse package >> > >>>>>> \begin{itemize} >> > >>>>>> \item Install the Mecurial plugin for Eclipse and >> then import the >> > >>>>>> PETSc repository to Eclipse. >> > >>>>>> \item elected New->Convert to C/C++ project and >> selected shared >> > >>>>>> library. After this point you can perform searchs >> in the code. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> A PETSc user has provided the following steps to >> build an Eclipse >> > >>>>>> index for PETSc that can be used with their own >> code without compiling PETSc >> > >>>>>> source into their project. >> > >>>>>> \begin{itemize} >> > >>>>>> \item In the user project source directory, create >> a symlink to the >> > >>>>>> petsc/src directory. >> > >>>>>> \item Refresh the project explorer in Eclipse, so >> the new symlink is >> > >>>>>> followed. >> > >>>>>> \item Right-click on the project in the project >> explorer, and choose >> > >>>>>> "Index -> Rebuild". The index should now be build. >> > >>>>>> \item Right-click on the PETSc symlink in the >> project explorer, and >> > >>>>>> choose "Exclude from build..." to make sure >> Eclipse does not try to compile >> > >>>>>> PETSc with the project. >> > >>>>>> \end{itemize} >> > >>>>>> >> > >>>>>> We'd love to have someone figure out how to do it >> right and include >> > >>>>>> that information. >> > >>>>>> >> > >>>>>> Barry >> > >>>>>> >> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >> > >>>>>> >> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >> Eclipse nicely generates >> > >>>>>> all my makefiles for me for my current project >> (which is written in C++). >> > >>>>>> I'd like to link PETSc w/my application but I'm >> not sure how to do this. >> > >>>>>> > >> > >>>>>> > Suggestions? >> > >>>>>> > >> > >>>>>> > Thanks, >> > >>>>>> > Matt >> > >>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > > >> > >> >> >> >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdbockma at ucsd.edu Wed Jul 27 21:22:56 2011 From: mdbockma at ucsd.edu (Matt Bockman) Date: Wed, 27 Jul 2011 19:22:56 -0700 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> <4E30B75E.2080805@imperial.ac.uk> <4E30BB54.6040106@imperial.ac.uk> Message-ID: Nevermind, I was completely wrong. I actually was able to build it using the commands that the original makefile produced (from PETSc), then when I went into Eclipse I forgot to press "clean"...derr. Matt On Wed, Jul 27, 2011 at 7:21 PM, Matt Bockman wrote: > Hi Berend, > > First of all, you have been extremely helpful. Second, I was FINALLY able > to compile the example through Eclipse. Sadly, it's the biggest breakthrough > I've had in the last 2 weeks of my research :(. > > Here's what I did: > > I tried running what Eclipse was running from a terminal: > > Eclipse: > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -O0 > -g3 -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP -MF"ex1.d" > -MT"ex1.d" -o"ex1.o" "../ex1.c" > > That did not work. So I ran make ex1 in the original ex1 tutorial directory > and saw what the makefile was doing there: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -o ex1.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex1.c > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -Wall -Wwrite-strings -Wno-strict-aliasing -g3 -o ex1 ex1.o > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 > -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > So apparently it was making an object file then linking. I then compared > the one from Eclipse and found that I was missing > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve. > Why I need this I do not know (ideas?). > > But, after I found that I added it to my Include Directories and it > compiled! > > Thanks Berend and others, I appreciate your persistence and aid. > > Matt > > > On Wed, Jul 27, 2011 at 6:28 PM, Berend van Wachem < > b.van-wachem at imperial.ac.uk> wrote: > >> Dear Matt, >> >> I can't directly see anything wrong with your settings. >> >> Have you tried compiling your example outside of eclipse with the PETSc >> makefile system? >> >> The previous pastebin output you emailed me does suggest that the error is >> during the compile stage: it cannot find the correct headers describing the >> implementations of PetscInitialize etc. >> >> Have you tried using the mpicc outside of eclipse directly on the >> directory? >> >> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/bin/**mpicc -I"/home/mdbockman/Documents/** >> Research/codes/petsc/petsc-3.**1-p8/include" >> -I"/home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >> 1-p8/linux-gnu-c-debug/**include" -O0 -g3 -Wall -Wwrite-strings >> -Wno-strict-aliasing -MMD -MP -MF"ex1.d" -MT"ex1.d" -o"ex1.o" "../ex1.c" >> >> within the eclipse directory? This way you can more easily play around >> with it to see what is the problem. >> >> I'm sorry I can't be of more help. >> >> Regards, >> Berend. >> >> >> >> On 07/28/2011 02:20 AM, Matt Bockman wrote: >> >>> Hi Berend, >>> >>> Under Library search path I have: >>> >>> /home/mdbockman/Documents/**Research/codes/petsc/petsc-3.** >>> 1-p8/linux-gnu-c-debug/lib >>> >>> For libraries I have: >>> >>> petsc >>> X11 >>> flapack >>> fblas >>> nsl >>> rt >>> m >>> dl >>> mpich >>> pthread >>> rt >>> gcc_s >>> mpichf90 >>> gfortran >>> m >>> m >>> dl >>> mpich >>> rt >>> gcc_s >>> dl >>> >>> I used the exact order as I found in petscmachineinfo.h file (see here: >>> http://pastebin.com/cqzNgjJe) >>> >>> Thank you very much for your help Berend, >>> Matt >>> >>> On Wed, Jul 27, 2011 at 6:11 PM, Berend van Wachem >>> >> >>> wrote: >>> >>> Dear Matt, >>> >>> >>> What do you have under >>> Properties->C/C++ Build -> Settings -> GCC Linker Libraries for both >>> "Libraries" and "Library Search Path"? >>> >>> Regards, >>> Berend. >>> >>> >>> >>> On 07/28/2011 02:04 AM, Matt Bockman wrote: >>> >>> Hi Berend, >>> >>> Under Properties->C/C++ Build -> Settings -> GCC C Compiler -> >>> Directories : Include paths (-I) I have: >>> >>> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >>> _1-p8/include >>> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >>> _1-p8/linux-gnu-c-debug/**include >>> >>> Under Properties -> C/C++ BUild -> Environment -> I have >>> >>> PETSC_ARCH = linux-gnu-c-debug >>> PETSC_DIR = >>> /home/mdbockman/Documents/__**Research/codes/petsc/petsc-3._** >>> _1-p8 >>> >>> In my source file I have included petsc.h: >>> >>> #include "petsc.h" >>> >>> Thanks, >>> Matt >>> >>> On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem >>> >> >>> > >>> >> >>> >>> wrote: >>> >>> Dear Matt, >>> >>> The error is definitely in the compiler stage: it cannot find that >>> appropriate petsc.h include file. >>> 1. Have you included petsc.h in the source file? >>> 2. What do you have under >>> -> Settings -> Includes >>> ? >>> >>> Also, just to make sure, you can set the appropriate values for >>> PETSC_DIR and PETSC_ARCH if you wish under >>> -> Settings -> Environment >>> >>> Regards, >>> >>> Berend. >>> >>> >>> >>> On 07/28/2011 01:49 AM, Matt Bockman wrote: >>> >>> Hi Berend, >>> >>> Here is the complete output of the build: >>> >>> http//pastebin.com/Es4ms4EF >>> >>> >>> >>> and by the 2nd to last line "collect2: ld returned 1 exit status" >>> I >>> believe it is failing during the linking step. I'm not 100% sure >>> though. >>> >>> Thanks for your quick response and help. Please advise, >>> >>> Matt >>> >>> On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem >>> >> ac.uk > >>> >> >>> >> >>> >> ____ac.uk >>> >> >>>> >>> wrote: >>> >>> Dear Matt, >>> >>> Does it say this during the compiling? Or linking? >>> >>> If it says this during the compiling, it means that eclipse cannot >>> find the PETSc header files. So, it must be the setting of the >>> "Includes". You might want to "hard-code" the directory, just to >>> make sure. >>> >>> It is indeed not completely straightforward - eclipse has so many >>> options. But trust me - many of them you will really learn to >>> appreciate over time. >>> >>> Kind regards, >>> >>> Berend. >>> >>> >>> >>> On 07/28/2011 01:34 AM, Matt Bockman wrote: >>> >>> Thanks Berend for your thorough response, >>> >>> I have done what you have said but I still get the same error >>> regarding >>> "undefined references to PetscInitialize" etc. It's like I didn't >>> include the petscksp.h file, but it's there. I even tried >>> petsc.h to no >>> avail. >>> >>> I'm not sure what the compiler is referring to when it says >>> "Undefined >>> references to ...". What I think this is is in the assembly code >>> generated by the compiler, there is a PetscInitialize symbol >>> that isn't >>> found in the library. But I'm soooooooooo confused at this point >>> :(. How >>> did you guys all learn how to compile this? >>> >>> Matt >>> >>> On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem >>> >> >>> > >>> >> >>> >> >>> >> ____ac.uk >>> >> >>> >>> >>> **. >>> >> **.__>____ac.uk >>> >>> >>> >> ____ac.uk >>> >> >>>>> >>> wrote: >>> >>> Dear Matt, >>> >>> I use Eclipse and have eclipse make the makefiles. >>> It is just a matter of indicating to eclipse where the PETSc >>> headers/libraries are to be found, so if you have a C project >>> which >>> needs PETSc headers and libraries: >>> >>> To do this, click on your managed C project with the right sided >>> mouse button, select >>> >>> Properties -> C/C++ Build -> Settings >>> >>> Then you get a new window with on the right hand side the various >>> setting options. >>> >>> Select Includes, and add the required PETSc paths. In my case I >>> have >>> added >>> ${PETSC_DIR}/include >>> ${PETSC_DIR}/${PETSC_ARCH}/___**_____include >>> >>> Then select "Libraries" under the header Linker >>> and you should set the Library search path: >>> ${PETSC_DIR}/${PETSC_ARCH}/lib >>> >>> and then the libraries, in my case: >>> m, petsc, stdc++, mpichxx, mpich, lapack, blas, gfortran, dl, >>> rt,gcc_s, pthread, X11 >>> (you can find these easily in >>> $PETSC_DIR/$PETSC_ARCH/_______**_petscmachineinfo.h) >>> >>> The nice thing is that in eclipse you can easily switch between >>> Debug/Release code, traverse into the PETSc source code etc. It's >>> really a very productive tool with PETSc I've found. >>> >>> Let me know if you have any questions. >>> >>> Kind regards, >>> >>> Berend. >>> >>> >>> >>> On 07/27/2011 11:25 PM, Matt Bockman wrote: >>> >>> Thanks everyone for the help, >>> >>> I was able to compile a single example in Eclipse using the >>> provided >>> makefile. I'm pretty new to makefiles so it's a LOT to digest. >>> I'm now >>> manually creating a makefile for my project in Eclipse (and I've >>> set >>> Eclipse up to use a makefile that I create instead of >>> automatically >>> generating one). Unfortunately this is a big pain but since I >>> can't >>> figure out how to make Eclipse automatically include a few files >>> in the >>> makefile I don't really have any other choices :(. >>> >>> Thanks again, >>> Matt >>> >>> On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>**>__> >>> wrote: >>> >>> There two problems(I think) in this code. >>> >>> 1) there is no main function in your source code. If this is the >>> only file you are compiling, you need to change the function name >>> to >>> main. >>> 2) linking should be done after object files are created. A simple >>> g++ call would first compile the main file and then link the >>> object >>> to the petsc lib i.e >>> >>> g++ -c -I($PETSC_INCLUDE) main.cpp >>> g++ -o main main.o $PETSC_LIBS >>> >>> alternatively, you could do it in a single line if you like >>> >>> g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS >>> >>> my point is you should link to petsc after compiling your own >>> code. >>> So wherever in Eclipse that you are seting the parameters, make >>> sure >>> the $PETSC_LIBS is in the linker option and not compiler. >>> >>> Mohammad >>> >>> >>> On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>>**> >>> wrote: >>> >>> I added the include directories from "make getincludedirs" and I >>> added the line from "make getlinklib". Eclipse creates a gcc >>> call as follows: >>> >>> /home/mdbockman/Documents/____**____Research/codes/petsc/** >>> petsc-__3.______1-p8/linux-**gnu-c-__debug/bin/______mpicc >>> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/linux-gnu-_**_c-debug/____include >>> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/include >>> -I/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/linux-gnu-_**_c-debug/____include >>> -O0 -g3 -pg -p -Wall >>> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >>> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >>> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >>> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >>> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >>> -lpetsc -lX11 >>> -Wl,-rpath,/home/mdbockman/___**_____Documents/Research/codes/** >>> ________petsc/petsc-3.1-p8/**linux-__gnu-______c-debug/lib >>> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >>> -lflapack -lfblas -lnsl -lrt -lm >>> -L/home/mdbockman/Documents/__**______Research/codes/petsc/__** >>> petsc-__3.____1-p8/linux-gnu-_**_c-debug/__lib >>> -L/usr/lib/x86_64-linux-gnu/__**______gcc/x86_64-linux-gnu/4.** >>> 5.2 >>> -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s >>> -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt >>> -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" >>> -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" >>> "../SparseMatrixPetsc.c >>> >>> And when it is compiled I get the following: >>> >>> http://pastebin.com/CbRzYcZj >>> >>> The source file which is being compiled is: >>> >>> http://pastebin.com/Q85hXvnS >>> >>> Please have a look. I'm not quite sure what I'm doing wrong but >>> I feel like I'm getting closer and closer to the solution. >>> >>> Matt >>> >>> >>> On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>>**> >>> wrote: >>> >>> use: >>> make getincludedirs >>> >>> Satish >>> >>> On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: >>> >>> > I applogize for the mistake; Include files are actually >>> located >>> > in $PETSC_DIR/include >>> > >>> > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >> >>>**>__>______wrote: >>> >>> >>> > >>> > > Ok then. Now I don't have enough experience with Eclipse so >>> > > I apologize beforehand if you already know these/have >>> tried them out. If >>> > > not, hopefully they can be of help. I assume there >>> should be a way in >>> > > Eclipse to give it the link lib directory. In plain >>> makefile that's just a >>> > > simple step when linking. To get all the needed >>> linklibs for petsc, you can >>> > > do >>> > > >>> > > make getlinklibs >>> > > >>> > > in the $PETSC_DIR. As for the needed include files, >>> they are all located >>> > > in >>> > > >>> > > $PETSC_DIR/$PETSC_ARCH/include >>> > > >>> > > Again, its easy to use these directories along with >>> your makefile. I'm not >>> > > sure about how you give them to Eclipse though. >>> Hopefully this has been >>> > > helpful. >>> > > >>> > > Best, >>> > > Mohammad >>> > > >>> > > >>> > > >>> > > >>> > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>>**> >>> wrote: >>> > > >>> > >> Just pointing it to the library would be sufficient. >>> > >> >>> > >> Matt >>> > >> >>> > >> >>> > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >> >>>**>__>______wrote: >>> >>> >>> > >> >>> > >>> So do you want to be able to compile PETSc with >>> Eclipse or just point it >>> > >>> to the library to use in your own applications? >>> > >>> >>> > >>> >>> > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>>**> >>> wrote: >>> > >>> >>> > >>>> Thanks Mohammad, >>> > >>>> >>> > >>>> I'll give that a shot. I use Qt Creator for some GUI >>> applications so I >>> > >>>> am familiar with it, but I've never tried doing a >>> non-Qt project in it. I'd >>> > >>>> really like to get Eclipse to work. >>> > >>>> >>> > >>>> Regarding the makefiles for eclipse. There are >>> makefiles that it >>> > >>>> generates (which are for GNU make) but I think I can >>> also manually create my >>> > >>>> makefiles. After sleeping on it, it seems like this >>> might be the best >>> > >>>> option, unless I can figure out a way to configure >>> eclipse to include the >>> > >>>> conf/variables and conf/rules files in the makefile. >>> > >>>> >>> > >>>> Matt >>> > >>>> >>> > >>>> >>> > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad Mirzadeh >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>>**> >>> >>> >>> > >>>> > wrote: >>> > >>>> >>> > >>>>> Although this is sort of orthogonal to what you do >>> right now, >>> > >>>>> I recommend Qt Creator as an alternative IDE to >>> Eclipse. It links nicely >>> > >>>>> with PETSc(or any other library for that matter) >>> and has excellent c/c++ >>> > >>>>> support. >>> > >>>>> >>> > >>>>> Mohammad >>> > >>>>> >>> > >>>>> >>> > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith >>> >>> > >>> >>> >> >>> >>> > >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> > >>> >>> >> >>>**>__>______wrote: >>> >>> >>> > >>>>> >>> > >>>>>> >>> > >>>>>> There is a tiny bit of information in the PETSc >>> users manual about >>> > >>>>>> Eclipse: >>> > >>>>>> >>> > >>>>>> \section{Eclipse Users} \sindex{eclipse} >>> > >>>>>> >>> > >>>>>> If you are interested in developing code that uses >>> PETSc from Eclipse >>> > >>>>>> or developing PETSc in Eclipse and have knowledge >>> of how to do indexing and >>> > >>>>>> build libraries in Eclipse please contact us at \ >>> > >>>>>> trl{petsc-dev at mcs.anl.gov >>> >>> > >>> >>> >>> >> >>> >> ____anl.gov >>> >> >>> >>> >>> **. >>> >> >>> >.__>____anl.gov >>> >>> >> >>> .>____anl.gov >>> >>> >>> >>>> >>> >>> >>> >>__. >>> >>> >>> > >>> >> >>> >>.__>____anl.gov >>> >>> >>> >>> >>> >>> >. >>> >> >>> >.>____anl.gov >>> >>> >>> >> ____mcs.anl.gov < >>> http://mcs.anl.gov> >>> >>> >> >>> >>>>>}. >>> >>> >>> >>> >>> >>> > >>>>>> >>> > >>>>>> To make PETSc an Eclipse package >>> > >>>>>> \begin{itemize} >>> > >>>>>> \item Install the Mecurial plugin for Eclipse and >>> then import the >>> > >>>>>> PETSc repository to Eclipse. >>> > >>>>>> \item elected New->Convert to C/C++ project and >>> selected shared >>> > >>>>>> library. After this point you can perform searchs >>> in the code. >>> > >>>>>> \end{itemize} >>> > >>>>>> >>> > >>>>>> A PETSc user has provided the following steps to >>> build an Eclipse >>> > >>>>>> index for PETSc that can be used with their own >>> code without compiling PETSc >>> > >>>>>> source into their project. >>> > >>>>>> \begin{itemize} >>> > >>>>>> \item In the user project source directory, create >>> a symlink to the >>> > >>>>>> petsc/src directory. >>> > >>>>>> \item Refresh the project explorer in Eclipse, so >>> the new symlink is >>> > >>>>>> followed. >>> > >>>>>> \item Right-click on the project in the project >>> explorer, and choose >>> > >>>>>> "Index -> Rebuild". The index should now be build. >>> > >>>>>> \item Right-click on the PETSc symlink in the >>> project explorer, and >>> > >>>>>> choose "Exclude from build..." to make sure >>> Eclipse does not try to compile >>> > >>>>>> PETSc with the project. >>> > >>>>>> \end{itemize} >>> > >>>>>> >>> > >>>>>> We'd love to have someone figure out how to do it >>> right and include >>> > >>>>>> that information. >>> > >>>>>> >>> > >>>>>> Barry >>> > >>>>>> >>> > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman wrote: >>> > >>>>>> >>> > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? >>> Eclipse nicely generates >>> > >>>>>> all my makefiles for me for my current project >>> (which is written in C++). >>> > >>>>>> I'd like to link PETSc w/my application but I'm >>> not sure how to do this. >>> > >>>>>> > >>> > >>>>>> > Suggestions? >>> > >>>>>> > >>> > >>>>>> > Thanks, >>> > >>>>>> > Matt >>> > >>>>>> >>> > >>>>>> >>> > >>>>> >>> > >>>> >>> > >>> >>> > >> >>> > > >>> > >>> >>> >>> >>> >>> >>> >>> >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 66152764 at qq.com Wed Jul 27 21:56:06 2011 From: 66152764 at qq.com (=?gbk?B?087K1rrDz9A=?=) Date: Thu, 28 Jul 2011 10:56:06 +0800 Subject: [petsc-users] [error] [petsc-3.1-p8] Message-ID: when i execute "make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test" ld: cannot find -lPEPCF90 make[3]: [ex5f] error1 (ignore) but in fact lPEPCF90 That library was from Intel Fortran 7.0 and earlier, as you note. That is no longer available. you can check the source : http://software.intel.com/en-us/forums/showthread.php?t=78989 ps: 1 -------------------------------- detail ------------------------------------------------------------- [chengwl at hksbs-s13 petsc-3.1-p8]$ make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test Running test examples to verify correct installation C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes --------------Error detected during compile or link!----------------------- See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html /opt/mpich2x-1.2.1p1/bin/mpif90 -c -g -I/home/chengwl/opt/petsc-3.1-p8/include -I/home/chengwl/opt/petsc-3.1-p8/include -I/opt/mpich2x-1.2.1p1/include -I/home/chengwl/opt/petsc-3.1-p8/include -I/opt/mpich2x-1.2.1p1/include -o ex5f.o ex5f.F /opt/mpich2x-1.2.1p1/bin/mpif90 -g -o ex5f ex5f.o -Wl,-rpath,/home/chengwl/opt/petsc-3.1-p8/lib -L/home/chengwl/opt/petsc-3.1-p8/lib -lpetsc -lX11 -Wl,-rpath,/opt/intel/composerxe-2011.4.191/mkl -L/opt/intel/composerxe-2011.4.191/mkl -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lPEPCF90 -ldl -L/opt/mpich2x-1.2.1p1/lib -lmpich -lopa -lpthread -lrt -L/opt/intel/composerxe-2011.4.191/compiler/lib/intel64 -L/opt/intel/composerxe-2011.4.191/mkl/lib/intel64 -L/opt/intel/Compiler/11.1/064/lib/intel64 -L/opt/intel/cce/10.1.018/lib -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -lmpichf90 -lifport -lifcore -lm -lm -ldl -lmpich -lopa -lpthread -lrt -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -ldl ld: cannot find -lPEPCF90 make[3]: [ex5f] error1 (ignore) /bin/rm -f ex5f.o Completed test examples 2 ---------------------------------- ./config/configure.py --prefix=/home/chengwl/o pt/petsc-3.1-p8 --with-mpi-dir=/opt/mpich2x-1.2.1p1/ --with-blas-lapack-dir=/opt /intel/composerxe-2011.4.191/mkl =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: alternateConfigureLibrary from PETSc.packages.petsc4py(config/PETSc/pac Compilers: C Compiler: /opt/mpich2x-1.2.1p1/bin/mpicc -g Fortran Compiler: /opt/mpich2x-1.2.1p1/bin/mpif90 -g Linkers: Static linker: /usr/bin/ar cr MPI: Includes: -I/opt/mpich2x-1.2.1p1/include X11: Includes: Library: -lX11 BLAS/LAPACK: -Wl,-rpath,/opt/intel/composerxe-2011.4.191/mkl -L/opt/intel/compos erxe-2011.4.191/mkl -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpth read PETSc: PETSC_ARCH: linux-gnu-intel PETSC_DIR: /home/chengwl/software/petsc-3.1-p8 Clanguage: C Memory alignment: 16 Scalar type: real Precision: double shared libraries: disabled dynamic libraries: disabled xxx=========================================================================xxx Configure stage complete. Now build PETSc libraries with: make PETSC_DIR=/home/chengwl/software/petsc-3.1-p8 PETSC_ARCH=linux-gnu-intel all xxx=========================================================================xxx it is odd. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Jul 27 22:30:27 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 27 Jul 2011 22:30:27 -0500 Subject: [petsc-users] [error] [petsc-3.1-p8] In-Reply-To: References: Message-ID: <17A8BF90-49A1-409F-93BE-E95AF41AA771@mcs.anl.gov> Edit $PETSC_DIR/$PETSC_ARCH/conf/petscvariables and remove all mention of -lPEPCF90 then run make test again Barry On Jul 27, 2011, at 9:56 PM, ???? wrote: > when i execute "make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test" > > ld: cannot find -lPEPCF90 > make[3]: [ex5f] error1 (ignore) > > but in fact lPEPCF90 That library was from Intel Fortran 7.0 and earlier, as you note. That is no longer available. > you can check the source : > http://software.intel.com/en-us/forums/showthread.php?t=78989 > > ps: > 1 -------------------------------- detail ------------------------------------------------------------- > [chengwl at hksbs-s13 petsc-3.1-p8]$ make PETSC_DIR=/home/chengwl/opt/petsc-3.1-p8 test > Running test examples to verify correct installation > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes > --------------Error detected during compile or link!----------------------- > See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html > /opt/mpich2x-1.2.1p1/bin/mpif90 -c -g -I/home/chengwl/opt/petsc-3.1-p8/include -I/home/chengwl/opt/petsc-3.1-p8/include -I/opt/mpich2x-1.2.1p1/include -I/home/chengwl/opt/petsc-3.1-p8/include -I/opt/mpich2x-1.2.1p1/include -o ex5f.o ex5f.F > /opt/mpich2x-1.2.1p1/bin/mpif90 -g -o ex5f ex5f.o -Wl,-rpath,/home/chengwl/opt/petsc-3.1-p8/lib -L/home/chengwl/opt/petsc-3.1-p8/lib -lpetsc -lX11 -Wl,-rpath,/opt/intel/composerxe-2011.4.191/mkl -L/opt/intel/composerxe-2011.4.191/mkl -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lPEPCF90 -ldl -L/opt/mpich2x-1.2.1p1/lib -lmpich -lopa -lpthread -lrt -L/opt/intel/composerxe-2011.4.191/compiler/lib/intel64 -L/opt/intel/composerxe-2011.4.191/mkl/lib/intel64 -L/opt/intel/Compiler/11.1/064/lib/intel64 -L/opt/intel/cce/10.1.018/lib -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -lmpichf90 -lifport -lifcore -lm -lm -ldl -lmpich -lopa -lpthread -lrt -limf -lsvml -lipgo -lirc -lgcc_s -lirc_s -ldl > ld: cannot find -lPEPCF90 > make[3]: [ex5f] error1 (ignore) > /bin/rm -f ex5f.o > Completed test examples > > 2 ---------------------------------- > > ./config/configure.py --prefix=/home/chengwl/o pt/petsc-3.1-p8 --with-mpi-dir=/opt/mpich2x-1.2.1p1/ --with-blas-lapack-dir=/opt /intel/composerxe-2011.4.191/mkl > =============================================================================== > Configuring PETSc to compile on your system > =============================================================================== > TESTING: alternateConfigureLibrary from PETSc.packages.petsc4py(config/PETSc/pac Compilers: > C Compiler: /opt/mpich2x-1.2.1p1/bin/mpicc -g > Fortran Compiler: /opt/mpich2x-1.2.1p1/bin/mpif90 -g > Linkers: > Static linker: /usr/bin/ar cr > MPI: > Includes: -I/opt/mpich2x-1.2.1p1/include > X11: > Includes: > Library: -lX11 > BLAS/LAPACK: -Wl,-rpath,/opt/intel/composerxe-2011.4.191/mkl -L/opt/intel/compos erxe-2011.4.191/mkl -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpth read > PETSc: > PETSC_ARCH: linux-gnu-intel > PETSC_DIR: /home/chengwl/software/petsc-3.1-p8 > Clanguage: C > Memory alignment: 16 > Scalar type: real > Precision: double > shared libraries: disabled > dynamic libraries: disabled > xxx=========================================================================xxx > Configure stage complete. Now build PETSc libraries with: > make PETSC_DIR=/home/chengwl/software/petsc-3.1-p8 PETSC_ARCH=linux-gnu-intel all > xxx=========================================================================xxx > > it is odd. From klaus.zimmermann at physik.uni-freiburg.de Thu Jul 28 00:36:18 2011 From: klaus.zimmermann at physik.uni-freiburg.de (Klaus Zimmermann) Date: Thu, 28 Jul 2011 07:36:18 +0200 Subject: [petsc-users] PETSc, C++ and Eclipse In-Reply-To: References: <4E30A342.90006@imperial.ac.uk> <4E30AFA9.8060500@imperial.ac.uk> <4E30B331.5090709@imperial.ac.uk> <4E30B75E.2080805@imperial.ac.uk> <4E30BB54.6040106@imperial.ac.uk> Message-ID: <4E30F552.2030600@physik.uni-freiburg.de> Hello Matt, "Undefined reference" is usually a linker error. As you noticed in the console normally a two step process is applied: First the .c file(s) is(are) compiled into .o files. Then they are linked. As Berend pointed out the error happened in the treatment of the .c file. So why is it linking at that stage? If you are using intel or gnu compiler it is because in the compile flags there is no -c. If you scan through the command that the original petcs makefile used you will find it for the compiling stage, but not in the eclipse version. Which compiler are you using? There are a lot of flags. Did you manually modify the compile command line? Is it possible you accidently removed -c? Regards, Klaus Am 7/28/11 4:22 AM, schrieb Matt Bockman: > Nevermind, I was completely wrong. I actually was able to build it using > the commands that the original makefile produced (from PETSc), then when > I went into Eclipse I forgot to press "clean"...derr. > > Matt > > On Wed, Jul 27, 2011 at 7:21 PM, Matt Bockman > wrote: > > Hi Berend, > > First of all, you have been extremely helpful. Second, I was FINALLY > able to compile the example through Eclipse. Sadly, it's the biggest > breakthrough I've had in the last 2 weeks of my research :(. > > Here's what I did: > > I tried running what Eclipse was running from a terminal: > > Eclipse: > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -O0 > -g3 -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP -MF"ex1.d" > -MT"ex1.d" -o"ex1.o" "../ex1.c" > > That did not work. So I ran make ex1 in the original ex1 tutorial > directory and saw what the makefile was doing there: > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -o ex1.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/include -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/include > -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex1.c > > /home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/bin/mpicc > -Wall -Wwrite-strings -Wno-strict-aliasing -g3 -o ex1 ex1.o > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lpetsc -lX11 > -Wl,-rpath,/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -lflapack -lfblas -lnsl -lrt -lm > -L/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/linux-gnu-c-debug/lib > -L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > So apparently it was making an object file then linking. I then > compared the one from Eclipse and found that I was missing > -I/home/mdbockman/Documents/Research/codes/petsc/petsc-3.1-p8/src/dm/mesh/sieve. > Why I need this I do not know (ideas?). > > But, after I found that I added it to my Include Directories and it > compiled! > > Thanks Berend and others, I appreciate your persistence and aid. > > Matt > > > On Wed, Jul 27, 2011 at 6:28 PM, Berend van Wachem > > > wrote: > > Dear Matt, > > I can't directly see anything wrong with your settings. > > Have you tried compiling your example outside of eclipse with > the PETSc makefile system? > > The previous pastebin output you emailed me does suggest that > the error is during the compile stage: it cannot find the > correct headers describing the implementations of > PetscInitialize etc. > > Have you tried using the mpicc outside of eclipse directly on > the directory? > > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/bin/__mpicc > -I"/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/include" > -I"/home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/__include" > -O0 -g3 -Wall -Wwrite-strings -Wno-strict-aliasing -MMD -MP > -MF"ex1.d" -MT"ex1.d" -o"ex1.o" "../ex1.c" > > within the eclipse directory? This way you can more easily play > around with it to see what is the problem. > > I'm sorry I can't be of more help. > > Regards, > Berend. > > > > On 07/28/2011 02:20 AM, Matt Bockman wrote: > > Hi Berend, > > Under Library search path I have: > > /home/mdbockman/Documents/__Research/codes/petsc/petsc-3.__1-p8/linux-gnu-c-debug/lib > > For libraries I have: > > petsc > X11 > flapack > fblas > nsl > rt > m > dl > mpich > pthread > rt > gcc_s > mpichf90 > gfortran > m > m > dl > mpich > rt > gcc_s > dl > > I used the exact order as I found in petscmachineinfo.h file > (see here: > http://pastebin.com/cqzNgjJe) > > Thank you very much for your help Berend, > Matt > > On Wed, Jul 27, 2011 at 6:11 PM, Berend van Wachem > > >> wrote: > > Dear Matt, > > > What do you have under > Properties->C/C++ Build -> Settings -> GCC Linker > Libraries for both > "Libraries" and "Library Search Path"? > > Regards, > Berend. > > > > On 07/28/2011 02:04 AM, Matt Bockman wrote: > > Hi Berend, > > Under Properties->C/C++ Build -> Settings -> GCC C > Compiler -> > Directories : Include paths (-I) I have: > > > /home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/include > > /home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8/linux-gnu-c-debug/__include > > Under Properties -> C/C++ BUild -> Environment -> I have > > PETSC_ARCH = linux-gnu-c-debug > PETSC_DIR = > > /home/mdbockman/Documents/____Research/codes/petsc/petsc-3.____1-p8 > > In my source file I have included petsc.h: > > #include "petsc.h" > > Thanks, > Matt > > On Wed, Jul 27, 2011 at 5:54 PM, Berend van Wachem > > > > ____ac.uk > >>> wrote: > > Dear Matt, > > The error is definitely in the compiler stage: it > cannot find that > appropriate petsc.h include file. > 1. Have you included petsc.h in the source file? > 2. What do you have under > -> Settings -> Includes > ? > > Also, just to make sure, you can set the appropriate > values for > PETSC_DIR and PETSC_ARCH if you wish under > -> Settings -> Environment > > Regards, > > Berend. > > > > On 07/28/2011 01:49 AM, Matt Bockman wrote: > > Hi Berend, > > Here is the complete output of the build: > > http//pastebin.com/Es4ms4EF > > > > > and by the 2nd to last line "collect2: ld returned 1 > exit status" I > believe it is failing during the linking step. I'm > not 100% sure > though. > > Thanks for your quick response and help. Please advise, > > Matt > > On Wed, Jul 27, 2011 at 5:39 PM, Berend van Wachem > > > > ____ac.uk > >> > . > .__>____ac.uk > > ____ac.uk > >>>> wrote: > > Dear Matt, > > Does it say this during the compiling? Or linking? > > If it says this during the compiling, it means that > eclipse cannot > find the PETSc header files. So, it must be the > setting of the > "Includes". You might want to "hard-code" the > directory, just to > make sure. > > It is indeed not completely straightforward - eclipse > has so many > options. But trust me - many of them you will really > learn to > appreciate over time. > > Kind regards, > > Berend. > > > > On 07/28/2011 01:34 AM, Matt Bockman wrote: > > Thanks Berend for your thorough response, > > I have done what you have said but I still get the > same error > regarding > "undefined references to PetscInitialize" etc. It's > like I didn't > include the petscksp.h file, but it's there. I even tried > petsc.h to no > avail. > > I'm not sure what the compiler is referring to when > it says > "Undefined > references to ...". What I think this is is in the > assembly code > generated by the compiler, there is a PetscInitialize > symbol > that isn't > found in the library. But I'm soooooooooo confused at > this point > :(. How > did you guys all learn how to compile this? > > Matt > > On Wed, Jul 27, 2011 at 4:46 PM, Berend van Wachem > > > > ____ac.uk > >> > . > .__>____ac.uk > > ____ac.uk > >>> > >__. > > >__.__>____ac.uk > > > > . > .__>____ac.uk > > ____ac.uk > >>>>> wrote: > > Dear Matt, > > I use Eclipse and have eclipse make the makefiles. > It is just a matter of indicating to eclipse where > the PETSc > headers/libraries are to be found, so if you have a C > project which > needs PETSc headers and libraries: > > To do this, click on your managed C project with the > right sided > mouse button, select > > Properties -> C/C++ Build -> Settings > > Then you get a new window with on the right hand side > the various > setting options. > > Select Includes, and add the required PETSc paths. In > my case I have > added > ${PETSC_DIR}/include > ${PETSC_DIR}/${PETSC_ARCH}/__________include > > Then select "Libraries" under the header Linker > and you should set the Library search path: > ${PETSC_DIR}/${PETSC_ARCH}/lib > > and then the libraries, in my case: > m, petsc, stdc++, mpichxx, mpich, lapack, blas, > gfortran, dl, > rt,gcc_s, pthread, X11 > (you can find these easily in > $PETSC_DIR/$PETSC_ARCH/__________petscmachineinfo.h) > > The nice thing is that in eclipse you can easily > switch between > Debug/Release code, traverse into the PETSc source > code etc. It's > really a very productive tool with PETSc I've found. > > Let me know if you have any questions. > > Kind regards, > > Berend. > > > > On 07/27/2011 11:25 PM, Matt Bockman wrote: > > Thanks everyone for the help, > > I was able to compile a single example in Eclipse > using the provided > makefile. I'm pretty new to makefiles so it's a LOT > to digest. > I'm now > manually creating a makefile for my project in > Eclipse (and I've set > Eclipse up to use a makefile that I create instead of > automatically > generating one). Unfortunately this is a big pain but > since I can't > figure out how to make Eclipse automatically include > a few files > in the > makefile I don't really have any other choices :(. > > Thanks again, > Matt > > On Wed, Jul 27, 2011 at 1:43 PM, Mohammad Mirzadeh > > > > >> > > > >>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > >>>>__>__> wrote: > > There two problems(I think) in this code. > > 1) there is no main function in your source code. If > this is the > only file you are compiling, you need to change the > function name to > main. > 2) linking should be done after object files are > created. A simple > g++ call would first compile the main file and then > link the object > to the petsc lib i.e > > g++ -c -I($PETSC_INCLUDE) main.cpp > g++ -o main main.o $PETSC_LIBS > > alternatively, you could do it in a single line if > you like > > g++ -o main -I($PETSC_INCLUDE) main.cpp $PETSC_LIBS > > my point is you should link to petsc after compiling > your own code. > So wherever in Eclipse that you are seting the > parameters, make sure > the $PETSC_LIBS is in the linker option and not compiler. > > Mohammad > > > On Wed, Jul 27, 2011 at 12:39 PM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>>__> > wrote: > > I added the include directories from "make > getincludedirs" and I > added the line from "make getlinklib". Eclipse > creates a gcc > call as follows: > > > /home/mdbockman/Documents/__________Research/codes/petsc/__petsc-__3.______1-p8/linux-__gnu-c-__debug/bin/______mpicc > > -I/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/linux-gnu-____c-debug/____include > > -I/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/include > > -I/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/linux-gnu-____c-debug/____include > -O0 -g3 -pg -p -Wall > > -Wl,-rpath,/home/mdbockman/__________Documents/Research/codes/__________petsc/petsc-3.1-p8/__linux-__gnu-______c-debug/lib > > -Wl,-rpath,/home/mdbockman/__________Documents/Research/codes/__________petsc/petsc-3.1-p8/__linux-__gnu-______c-debug/lib > > -L/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/linux-gnu-____c-debug/__lib > -lpetsc -lX11 > > -Wl,-rpath,/home/mdbockman/__________Documents/Research/codes/__________petsc/petsc-3.1-p8/__linux-__gnu-______c-debug/lib > > -L/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/linux-gnu-____c-debug/__lib > -lflapack -lfblas -lnsl -lrt -lm > > -L/home/mdbockman/Documents/__________Research/codes/petsc/____petsc-__3.____1-p8/linux-gnu-____c-debug/__lib > > -L/usr/lib/x86_64-linux-gnu/__________gcc/x86_64-linux-gnu/4.__5.2 > -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread > -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lpthread -lrt > -lgcc_s -ldl -MMD -MP -MF"SparseMatrixPetsc.d" > -MT"SparseMatrixPetsc.d" -o"SparseMatrixPetsc.o" > "../SparseMatrixPetsc.c > > And when it is compiled I get the following: > > http://pastebin.com/CbRzYcZj > > The source file which is being compiled is: > > http://pastebin.com/Q85hXvnS > > Please have a look. I'm not quite sure what I'm doing > wrong but > I feel like I'm getting closer and closer to the > solution. > > Matt > > > On Wed, Jul 27, 2011 at 11:52 AM, Satish Balay > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>>__> > wrote: > > use: > make getincludedirs > > Satish > > On Wed, 27 Jul 2011, Mohammad Mirzadeh wrote: > > > I applogize for the mistake; Include files are > actually > located > > in $PETSC_DIR/include > > > > On Wed, Jul 27, 2011 at 11:18 AM, Mohammad Mirzadeh > > > > >> > > > >>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > > >>>>__>__>______wrote: > > > > > > > Ok then. Now I don't have enough experience with > Eclipse so > > > I apologize beforehand if you already know > these/have > tried them out. If > > > not, hopefully they can be of help. I assume there > should be a way in > > > Eclipse to give it the link lib directory. In plain > makefile that's just a > > > simple step when linking. To get all the needed > linklibs for petsc, you can > > > do > > > > > > make getlinklibs > > > > > > in the $PETSC_DIR. As for the needed include files, > they are all located > > > in > > > > > > $PETSC_DIR/$PETSC_ARCH/include > > > > > > Again, its easy to use these directories along with > your makefile. I'm not > > > sure about how you give them to Eclipse though. > Hopefully this has been > > > helpful. > > > > > > Best, > > > Mohammad > > > > > > > > > > > > > > > On Wed, Jul 27, 2011 at 10:52 AM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>>__> > wrote: > > > > > >> Just pointing it to the library would be > sufficient. > > >> > > >> Matt > > >> > > >> > > >> On Wed, Jul 27, 2011 at 10:21 AM, Mohammad Mirzadeh > > > > >> > > > >>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > > >>>>__>__>______wrote: > > > > >> > > >>> So do you want to be able to compile PETSc with > Eclipse or just point it > > >>> to the library to use in your own applications? > > >>> > > >>> > > >>> On Wed, Jul 27, 2011 at 9:14 AM, Matt Bockman > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>>>__> > wrote: > > >>> > > >>>> Thanks Mohammad, > > >>>> > > >>>> I'll give that a shot. I use Qt Creator for > some GUI > applications so I > > >>>> am familiar with it, but I've never tried doing a > non-Qt project in it. I'd > > >>>> really like to get Eclipse to work. > > >>>> > > >>>> Regarding the makefiles for eclipse. There are > makefiles that it > > >>>> generates (which are for GNU make) but I > think I can > also manually create my > > >>>> makefiles. After sleeping on it, it seems > like this > might be the best > > >>>> option, unless I can figure out a way to > configure > eclipse to include the > > >>>> conf/variables and conf/rules files in the > makefile. > > >>>> > > >>>> Matt > > >>>> > > >>>> > > >>>> On Wed, Jul 27, 2011 at 12:01 AM, Mohammad > Mirzadeh > > > > >> > > > >>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > >>>>__> > > > > >>>> > wrote: > > >>>> > > >>>>> Although this is sort of orthogonal to what > you do > right now, > > >>>>> I recommend Qt Creator as an alternative IDE to > Eclipse. It links nicely > > >>>>> with PETSc(or any other library for that matter) > and has excellent c/c++ > > >>>>> support. > > >>>>> > > >>>>> Mohammad > > >>>>> > > >>>>> > > >>>>> On Tue, Jul 26, 2011 at 7:22 PM, Barry Smith > > > > >> > > > >>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > > >>>>__>__>______wrote: > > > > >>>>> > > >>>>>> > > >>>>>> There is a tiny bit of information in the PETSc > users manual about > > >>>>>> Eclipse: > > >>>>>> > > >>>>>> \section{Eclipse Users} \sindex{eclipse} > > >>>>>> > > >>>>>> If you are interested in developing code > that uses > PETSc from Eclipse > > >>>>>> or developing PETSc in Eclipse and have > knowledge > of how to do indexing and > > >>>>>> build libraries in Eclipse please contact > us at \ > > >>>>>> trl{petsc-dev at mcs.anl.gov > > > > ____anl.gov > >> > . > .__>____anl.gov > > ____anl.gov > >>> > >__. > > >.__>____anl.gov > > > . > .>____anl.gov > > ____mcs.anl.gov > > >>>> > > > > >>__. > > > > > ____mcs > >>.__>____anl.gov > > > > > > > >. > ____mcs > >.>____anl.gov > > > > > __>____mcs.anl.gov > > > ____mcs.anl.gov > > >>>>>}. > > > > > > > >>>>>> > > >>>>>> To make PETSc an Eclipse package > > >>>>>> \begin{itemize} > > >>>>>> \item Install the Mecurial plugin for > Eclipse and > then import the > > >>>>>> PETSc repository to Eclipse. > > >>>>>> \item elected New->Convert to C/C++ project and > selected shared > > >>>>>> library. After this point you can perform > searchs > in the code. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> A PETSc user has provided the following > steps to > build an Eclipse > > >>>>>> index for PETSc that can be used with their own > code without compiling PETSc > > >>>>>> source into their project. > > >>>>>> \begin{itemize} > > >>>>>> \item In the user project source directory, > create > a symlink to the > > >>>>>> petsc/src directory. > > >>>>>> \item Refresh the project explorer in > Eclipse, so > the new symlink is > > >>>>>> followed. > > >>>>>> \item Right-click on the project in the project > explorer, and choose > > >>>>>> "Index -> Rebuild". The index should now be > build. > > >>>>>> \item Right-click on the PETSc symlink in the > project explorer, and > > >>>>>> choose "Exclude from build..." to make sure > Eclipse does not try to compile > > >>>>>> PETSc with the project. > > >>>>>> \end{itemize} > > >>>>>> > > >>>>>> We'd love to have someone figure out how to > do it > right and include > > >>>>>> that information. > > >>>>>> > > >>>>>> Barry > > >>>>>> > > >>>>>> On Jul 26, 2011, at 4:32 PM, Matt Bockman > wrote: > > >>>>>> > > >>>>>> > Has anyone gotten PETSc to work w/Eclipse? > Eclipse nicely generates > > >>>>>> all my makefiles for me for my current project > (which is written in C++). > > >>>>>> I'd like to link PETSc w/my application but I'm > not sure how to do this. > > >>>>>> > > > >>>>>> > Suggestions? > > >>>>>> > > > >>>>>> > Thanks, > > >>>>>> > Matt > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > > > > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4872 bytes Desc: S/MIME Kryptografische Unterschrift URL: From jchludzinski at gmail.com Thu Jul 28 00:55:31 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Thu, 28 Jul 2011 01:55:31 -0400 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: You have: *typedef enum {DIRICHLET, NEUMANN} BCType;* twice in your source: ex29.c ---John On Wed, Jul 27, 2011 at 3:46 PM, Alan Wei wrote: > Dear John, > Thanks for your quick reply. I think the error message says that the > BCType has some problem. However, it is originally there and I have not > changed it at all. Is that because I define it in ComputeRHS again and let > ex29.c include ComputeRHS.h so the BCtype is duplicated? > > thanks, > Alan > > On Wed, Jul 27, 2011 at 2:04 PM, John Chludzinski > wrote: >> >> Without wading through the attached zip file, it looks like you're missing >> some typedef's that come from a missing include file. ---John >> >> On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei wrote: >>> >>> Dear Sir/Madam, >>> I hope you're having a nice day. >>> I asked several questions about src/ksp/ksp/example/tutorial/ex29.c >>> before, and learned some beginning staff of PETSc. Right now, I want to >>> start to use it to my own program. Firstly, I want to isolate the ComputeRHS >>> function in ex29, which means I want to put ComputeRHS to another c program >>> file. I did that; however, some problem pop up when I compile it. >>> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >>> ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas >>> -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c >>> ex29.c: In function ComputeRHS: >>> ex29.c:39: error: storage class specified for parameter BCType >>> ex29.c:44: error: expected specifier-qualifier-list before BCType >>> ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { token >>> ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { token >>> ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { token >>> ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { token >>> ex29.c:220: error: old-style parameter declarations in prototyped >>> function definition >>> ex29.c:220: error: expected { at end of input >>> make: [ex29.o] Error 1 (ignored) >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >>> ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>> -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -Wall >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 >>> ex29.o ComputeRHS.o >>> -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lpetsc >>> -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >>> -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl -lmpich >>> -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl >>> -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl >>> gcc: ex29.o: No such file or directory >>> make: [ex29] Error 1 (ignored) >>> >>> The program and modified makefile are attached. Could you please take a >>> look and give me some suggestions. >>> >>> best, >>> Alan >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Thu Jul 28 10:01:52 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Thu, 28 Jul 2011 09:01:52 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: <128187803.160591.1311616691759.JavaMail.root@zimbra.anl.gov> <3F2C83C9-3E63-458A-8AA8-534F54729A57@mcmillan-mcgee.com> Message-ID: <407CFB56-B9B1-4F21-8680-8E20049C072E@mcmillan-mcgee.com> Hi Barry, Shri, I reworked my implementation to include only the time-independent electrical problem, and to make use of SNESVISetVariableBounds(). However, it does not work quite as I expected: Constraints on variables that are functions of other variables have no effect on the variables by which the functions are defined. My constraints are set on the four degrees of freedom as follows: -SNES_VI_INF <= V_block <= SNES_VI_INF -SNES_VI_INF <= V_electrode <= SNES_VI_INF 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) The residuals are set as follows, and the Jacobian entries are set correspondingly: f[V_block] = {function of adjacent V_blocks, V_electrode} f[V_electrode] = 0.0 + 0.0i f[magnitude(V_electrode)] = 0.0 + 0.0i f[magnitude(I_electrode)] = 0.0 + 0.0i The first two potentials, V_block and V_electrode, are independent, while magnitude(V_electrode) is a function of V_electrode only and the electrode current magnitude(I_electrode) is a function of V_block and V_electrode. Note that V_electrode acts as a source term in the finite difference formulation for V_block. While all of these variables are complex, the latter two always have zero imaginary parts. I suppose I am assuming that PETSc knows to compare only the real parts if the imaginary parts are zero. (Is this a bad assumption?) When solving with PETSc (using -snes_vi_type rs) V_electrode stays at its maximum, as set in the initial guess, and the V_block distribution falls properly from that. However, measurements of magnitude(I_electrode) are way above maximum and do not move downwards with successive SNES iterations. I can also set the constraint on magnitude(V_electrode) to less than maximum and it does not affect the value of V_electrode. How can I tell PETSc to change V_electrode when the magnitude(V_electrode) or magnitude(I_electrode) constraints are not met? Thanks again for your help. Jon On 2011-07-25, at 8:58 PM, Barry Smith wrote: > > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > >> Hi Shri, >> >> Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. >> >> For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? >> > > You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. > > Barry > >> Thank you again, >> >> Jon >> >> On 2011-07-25, at 11:58 AM, Shri wrote: >> >>> >>> ----- Original Message ----- >>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>>> >>>>> Barry, >>>>> >>>>> Thank you so much for your response. Lucky, indeed! I look forward >>>>> to trying out these new features. >>>>> >>>>> I neglected to mention in my original post that my electrical >>>>> problem is part of a DAE, which includes a time-dependent heating >>>>> problem. Can SNESVI constraints be used in conjunction with >>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only >>>>> need the constraints for the time-independent electrical portion.) >>>> >>>> We have not yet put that in but Shri is starting to look at that just >>>> now. Basically we would have a TSVISetVariableBounds() and handle >>>> everything from there. I suggest you start with a simple time >>>> electrical portion with constraints to explore and we'll go from >>>> there. Shri is actually a electrical networks guy and so can speak >>>> your language. >>> >>> >>> I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. >>> A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). >>> i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). >>> ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space >>> (-snes_vi_type rs). >>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. >>> >>> By the way, does your problem have bounds on the variables or bounds on some function of the variables? >>> >>> Shri >>> >>> >>> >>>> >>>> Barry >>>> >>>> >>>> >>>>> >>>>> Thank you again, >>>>> >>>>> Jon >>>>> >>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>>> >>>>>> >>>>>> Jon, >>>>>> >>>>>> You may be in luck. In PETSc-dev >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>>> have now implemented variational inequality non-linear solvers >>>>>> with box constraints. >>>>>> >>>>>> That is one has a set of variables u and algebraic equations >>>>>> F(u) = 0 (say of size n each) but in addition one has >>>>>> constraints lu <= u <= uu where some or all of lu may be >>>>>> negative infinity (no constraints) and some or all of uu may be >>>>>> infinity (no constraints). There is also a constraint on the >>>>>> sign of F() for those equations associated with active >>>>>> constraints. If your constraints are not in this form sometimes >>>>>> you can introduce new additional variables to get it in this >>>>>> form. Read up a little on variational inequalities on the web. >>>>>> >>>>>> To use this you provide the usual SNES function and Jacobian but >>>>>> you also provide SNESVISetVariableBounds() there is a manual >>>>>> page for this function and for for SNESVI at >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>>> (the link is broken right now but Satish is fixing it). Plus >>>>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>>>> >>>>>> This is all new code so we would be interested in your feedback. >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am trying to add a constraint feature to my first PETSc >>>>>>> application, which uses the finite difference method to calculate >>>>>>> the potential distribution produced by a collection of electrodes >>>>>>> in a resistive medium. I would like to make this simulation more >>>>>>> realistic by imposing a maximum electric current and a maximum >>>>>>> potential difference that can be supplied to each electrode by the >>>>>>> power supply. If the medium between the electrodes is very >>>>>>> conductive, the current maximum would be exceeded by the maximum >>>>>>> potential difference, so the potential difference should be >>>>>>> decreased from maximum until it produces the maximum current. On >>>>>>> the other hand, the potential difference between the electrodes >>>>>>> should remain at maximum as long as the current remains below >>>>>>> maximum (say, for a less conductive medium). >>>>>>> >>>>>>> I added an extra degree of freedom (the electrode voltages) to my >>>>>>> DMDA, and I developed a set of conditional expressions that >>>>>>> describe the above constraints, but one problem is that the logic >>>>>>> relies on if-then-else decisions that are made when forming the >>>>>>> function/residual and the Jacobian. Once these decisions are made, >>>>>>> of course, the conditions are not checked again until the next >>>>>>> function or Jacobian evaluation. The non-linear solver then tends >>>>>>> to oscillate between extreme solutions to the opposing conditions >>>>>>> with each iteration, and never converges towards a reasonable >>>>>>> solution. >>>>>>> >>>>>>> Is there a better strategy for solving such problems? Does PETSc >>>>>>> offer mechanisms to aid in their solution? I would very much >>>>>>> appreciate any hints. >>>>>>> >>>>>>> Thank you for your time, >>>>>>> >>>>>>> Jon >>>>>> >>>>> >>> >> > From zhenglun.wei at gmail.com Thu Jul 28 10:22:51 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Thu, 28 Jul 2011 10:22:51 -0500 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: Thanks, John. However, on my opinion, one of them has been commented out. ^_^ best, Alan On Thu, Jul 28, 2011 at 12:55 AM, John Chludzinski wrote: > You have: > > *typedef enum {DIRICHLET, NEUMANN} BCType;* > > twice in your source: ex29.c > > ---John > > > On Wed, Jul 27, 2011 at 3:46 PM, Alan Wei wrote: > > Dear John, > > Thanks for your quick reply. I think the error message says that the > > BCType has some problem. However, it is originally there and I have not > > changed it at all. Is that because I define it in ComputeRHS again and > let > > ex29.c include ComputeRHS.h so the BCtype is duplicated? > > > > thanks, > > Alan > > > > On Wed, Jul 27, 2011 at 2:04 PM, John Chludzinski < > jchludzinski at gmail.com> > > wrote: > >> > >> Without wading through the attached zip file, it looks like you're > missing > >> some typedef's that come from a missing include file. ---John > >> > >> On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei > wrote: > >>> > >>> Dear Sir/Madam, > >>> I hope you're having a nice day. > >>> I asked several questions about src/ksp/ksp/example/tutorial/ex29.c > >>> before, and learned some beginning staff of PETSc. Right now, I want to > >>> start to use it to my own program. Firstly, I want to isolate the > ComputeRHS > >>> function in ex29, which means I want to put ComputeRHS to another c > program > >>> file. I did that; however, some problem pop up when I compile it. > >>> > >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o > >>> ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas > >>> -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include > >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include > >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve > >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c > >>> ex29.c: In function ComputeRHS: > >>> ex29.c:39: error: storage class specified for parameter BCType > >>> ex29.c:44: error: expected specifier-qualifier-list before BCType > >>> ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { token > >>> ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { > token > >>> ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { > token > >>> ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { > token > >>> ex29.c:220: error: old-style parameter declarations in prototyped > >>> function definition > >>> ex29.c:220: error: expected { at end of input > >>> make: [ex29.o] Error 1 (ignored) > >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o > >>> ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing > >>> -Wno-unknown-pragmas -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include > >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include > >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve > >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c > >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc > -Wall > >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 > >>> ex29.o ComputeRHS.o > >>> -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib -lpetsc > >>> -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib > >>> -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl > -lmpich > >>> -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl > >>> -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl > >>> gcc: ex29.o: No such file or directory > >>> make: [ex29] Error 1 (ignored) > >>> > >>> The program and modified makefile are attached. Could you please take a > >>> look and give me some suggestions. > >>> > >>> best, > >>> Alan > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Thu Jul 28 11:21:15 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 28 Jul 2011 11:21:15 -0500 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: <4E305497.1070405@tuhh.de> References: <4E2F2950.2030701@tuhh.de> <4E305497.1070405@tuhh.de> Message-ID: Ping : I tested similar calling procedure using petsc-dev/src/ksp/ksp/examples/tutorials/ex5.c successfully. Try following: 1) switch to petsc-dev. See http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html on how to get it. 2) send us a short and stand-alone code that reproduce the error. We can investigate it. Hong > Dear Hong, > > thank you very much for your time. Can you tell me when should > ?KSPSetFromOptions(ksp) be called? > well, libMesh has a class called "petsc_linear_solver", in its original code > KSPSetFromOptions(ksp) is called while the object is created. I couldn't get > any output, when I run the program with the option > > ? -pc_type ilu -pc_factor_mat_solver_package superlu > -mat_superlu_ilu_droptol 0.1 -ksp_view > > The actual solve() function of the "petsc_linear_solver" contains the > similar code as in the ex2.c > ... > ? ?ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? this->same_preconditioner ? > SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); > ? ?CHKERRABORT(libMesh::COMM_WORLD,ierr); > > ?// Set the tolerances for the iterative solver. ?Use the user-supplied > ?// tolerance for the relative residual & leave the others at default > values. > ?ierr = KSPSetTolerances (_ksp, tol, PETSC_DEFAULT, PETSC_DEFAULT, max_its); > ?CHKERRABORT(libMesh::COMM_WORLD,ierr); > > ?ierr = KSPSetFromOptions (_ksp); <-------------- ?I added these two lines, > ?CHKERRABORT(libMesh::COMM_WORLD,ierr); <-------------- ?then I got proper > output. > > ?// Solve the linear system > ?ierr = KSPSolve (_ksp, rhs->vec(), solution->vec()); > ?CHKERRABORT(libMesh::COMM_WORLD,ierr); > .... > > The problem is that I have to solve a multiple frequency system, and for > each frequency I have several RHS. The same ksp context is used to solve the > systems repeatedly. I always call KSPSetOperators() with the option > DIFFERENT_NONZERO_PATTERN for the first RHS and SAME_PRECONDITIONER for the > rest, since the system matrix remains the same. ?For the first frequency, > everything is fine, both DIFFERENT_NONZERO_PATTERN and SAME_PRECONDITIONER > options. I got following output from the -ksp_view. > ... > PC Object: > ?type: ilu > ? ?ILU: out-of-place factorization > ? ?0 levels of fill > ? ?tolerance for zero pivot 1e-12 > ? ?using diagonal shift to prevent zero pivot > ? ?matrix ordering: natural > ? ?factor fill ratio given 0, needed 0 > ? ? ?Factored matrix follows: > ? ? ? ?Matrix Object: > ? ? ? ? ?type=seqaij, rows=2883, cols=2883 > ? ? ? ? ?package used to perform factorization: superlu <---------------- > superlu is properly set > .... > > but when the second frequency comes up, KSPSetOperators() is called again > with DIFFERENT_NONZERO_PATTERN option for the first RHS. the command line > option doesn't seem to work anymore. > ... > PC Object: > ?type: ilu > ? ?ILU: out-of-place factorization > ? ?0 levels of fill > ? ?tolerance for zero pivot 1e-12 > ? ?using diagonal shift to prevent zero pivot > ? ?matrix ordering: natural > ? ?factor fill ratio given 1, needed 1 > ? ? ?Factored matrix follows: > ? ? ? ?Matrix Object: > ? ? ? ? ?type=seqaij, rows=2883, cols=2883 > ? ? ? ? ?package used to perform factorization: petsc <------------ not > superlu anymore > ... > > and then for the second RHS, when it calls KSPSetOperators(ksp) in the > solve() routine, petsc throws the following error message. > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Object is in wrong state! > [0]PETSC ERROR: Cannot change solver matrix package after PC has been setup > or used! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 > CDT 2011 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./PLTMainTest.opt on a linux-gnu named abe-vm by mubpr Wed > Jul 27 19:41:22 2011 > [0]PETSC ERROR: Libraries linked from > /home/mubpr/libs/petsc/linux-gnu-acml-shared-opt/lib > [0]PETSC ERROR: Configure run at Tue Jul 26 22:09:39 2011 > [0]PETSC ERROR: Configure options > --with-blas-lapack-lib=/home/mubpr/libs/acml/gfortran64_mp/lib/libacml_mp.so > --with-blas-lapack-include=/home/mubpr/libs/acml/gfortran64_mp/include > --with-scalar-type=complex --with-clanguage=c++ > --with-mpi-dir=/home/mubpr/libs/mpich2 --download-superlu=yes > --with-superlu=1 --download-parmetis=yes --with-parmetis=1 > --download-superlu_dist=yes --with-superlu_dist=1 --download-mumps=yes > --with-mumps=1 --download-blacs=yes --with-blacs=1 --download-scalapack=yes > --with-scalapack=1 --download-spooles=yes --with-spooles=1 > --download-umfpack=yes --with-umfpack=1 --with-debugging=no --with-shared=1 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: PCFactorSetMatSolverPackage_Factor() line 183 in > src/ksp/pc/impls/factor/factimpl.c > [0]PETSC ERROR: PCFactorSetMatSolverPackage() line 276 in > src/ksp/pc/impls/factor/factor.c > [0]PETSC ERROR: PCSetFromOptions_Factor() line 275 in > src/ksp/pc/impls/factor/factimpl.c > [0]PETSC ERROR: PCSetFromOptions_ILU() line 112 in > src/ksp/pc/impls/factor/ilu/ilu.c > [0]PETSC ERROR: PCSetFromOptions() line 185 in src/ksp/pc/interface/pcset.c > [0]PETSC ERROR: KSPSetFromOptions() line 323 in src/ksp/ksp/interface/itcl.c > [0]PETSC ERROR: User provided function() line 696 in > "unknowndirectory/"src/solvers/petsc_linear_solver.C > > Any idea what the problem may be? I would be very grateful, if you can give > me any hint. thanks alot > > best regards > Ping > > > > Am 27.07.2011 04:13, schrieb Hong Zhang: >> >> Did you call KSPSetFromOptions(ksp)? >> Run your code with '-options_table' to dump list of options inputted >> or '-options_left' to dump list of unused options. >> >> I tested with petsc-3.1/src/ksp/ksp/examples/tutorials/ex2.c: >> ./ex2 >> ... >> ? ? ? ? ? ? SuperLU run parameters: >> ? ? ? ? ? ? ... >> ? ? ? ? ? ? ? ILU_DropTol: 0.1 >> ? ? ? ? ? ? ? ILU_FillTol: 0.01 >> ? ? ? ? ? ? ? ILU_FillFactor: 10 >> ? ? ? ? ? ? ? ILU_DropRule: 9 >> ? ? ? ? ? ? ? ILU_Norm: 2 >> ? ? ? ? ? ? ? ILU_MILU: 2 >> >> Hong >> >> On Tue, Jul 26, 2011 at 3:53 PM, Ping Rong ?wrote: >>> >>> Dear developers, >>> >>> I have compiled petsc-3.1-p8 for a while. Now I would like to use superlu >>> as >>> an ilu preconditioner, since it offers the drop tolerance option. I have >>> read in a thread >>> >>> (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) >>> that one can run the code with option >>> >>> -pc_type ilu -pc_factor_mat_solver_package superlu >>> -mat_superlu_ilu_droptol >>> <> >>> >>> to setup the ilu preconditioner. I also use the option "-help |grep >>> superlu" >>> to check the settings. However, no matter how I change the value of >>> -mat_superlu_ilu_droptol, I always got the same result >>> ... >>> ?-mat_superlu_lwork<0>: size of work array in bytes used by factorization >>> (None) >>> ?-mat_superlu_ilu_droptol<0.0001>: ILU_DropTol (None) >>> ?-mat_superlu_ilu_filltol<0.01>: ILU_FillTol (None) >>> ... >>> I dont know whether I understand it correctly. But it seems to me the >>> value >>> of the droptol has never been changed. In that maillist thread, it was >>> also >>> mentioned that the dev-version is recommended, because of some bugs in >>> the >>> superlu interface. I have then compiled the current dev-version. but the >>> problem is my program is based on another library (libMesh) which uses >>> petsc >>> as a solver package. Since some of the interfaces have been changed in >>> the >>> dev-version, I would not be able to compile the libMesh with petsc >>> anymore. >>> Is there anyway I can correct the superlu interface in the 3.1-p8 version >>> directly? Any help will be appreciated!! >>> >>> Best regards >>> >>> -- >>> Ping Rong, M.Sc. >>> Hamburg University of Technology >>> Institut of modelling and computation >>> Denickestra?e 17 (Room 3031) >>> 21073 Hamburg >>> >>> Tel.: ++49 - (0)40 42878 2749 >>> Fax: ?++49 - (0)40 42878 43533 >>> Email: ping.rong at tuhh.de >>> >>> > > > -- > Ping Rong, M.Sc. > Hamburg University of Technology > Institut of modelling and computation > Denickestra?e 17 (Room 3031) > 21073 Hamburg > > Tel.: ++49 - (0)40 42878 2749 > Fax: ?++49 - (0)40 42878 43533 > Email: ping.rong at tuhh.de > > From adam1.byrd at gmail.com Thu Jul 28 12:44:32 2011 From: adam1.byrd at gmail.com (Adam Byrd) Date: Thu, 28 Jul 2011 13:44:32 -0400 Subject: [petsc-users] Retrieving Individual Values from a Parallel Matrix Message-ID: Dear All, I'm attempting to find the most efficient/simple way to collectively retrieve just a handful of values (local and non local) from a parallel matrix. I'm filling two 4x4 arrays with 4 values from 8 different rows of the solution matrix of a previous solve. So far, MatGetSubMatrices looks my only option (MatGetValues and MatGetRow can only get local values), but seems unlikely to be the most efficient way to do this unless I am able to change the row and column ordering when creating the submatrix. It appears to retain the relative ordering of the rows and columns from the original matrix. Essentially, I'm trying to do this: myArray[n][m] = myPetscMat[i][j] This is the code I'm rewriting to work collectively on a parallel matrix: for(int i=0; i<4; i++) { ierr = MatGetRow(inverseHamiltonian, isl[i]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); for(int j=0; j<4; j++) { gaml[i][j] = rowValues[isr[j]-1]; } //endfor ierr = MatRestoreRow(inverseHamiltonian, isl[i]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); } //endfor for(int j=0; j<4; j++) { ierr = MatGetRow(inverseHamiltonian, isr[j]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); for(int i=0; i<4; i++) { gamrr[i][j] = conj(rowValues[isl[i]-1]); } //endfor ierr = MatRestoreRow(inverseHamiltonian, isr[j]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); } //endfor where isr[] and isl[] contain arbitrary, unsorted indices. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Thu Jul 28 13:42:52 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Thu, 28 Jul 2011 14:42:52 -0400 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: In ComputeRHS.h use: #ifndef _COMPURERHS_H #define _COMPURERHS_H PetscErrorCode ComputeRHS(DMMG dmmg, Vec b); #endif ---John On Thu, Jul 28, 2011 at 11:22 AM, Alan Wei wrote: > Thanks, John. However, on my opinion, one of them has been commented out. > ^_^ > > best, > Alan > > > On Thu, Jul 28, 2011 at 12:55 AM, John Chludzinski > wrote: > >> You have: >> >> *typedef enum {DIRICHLET, NEUMANN} BCType;* >> >> twice in your source: ex29.c >> >> ---John >> >> >> On Wed, Jul 27, 2011 at 3:46 PM, Alan Wei wrote: >> > Dear John, >> > Thanks for your quick reply. I think the error message says that the >> > BCType has some problem. However, it is originally there and I have not >> > changed it at all. Is that because I define it in ComputeRHS again and >> let >> > ex29.c include ComputeRHS.h so the BCtype is duplicated? >> > >> > thanks, >> > Alan >> > >> > On Wed, Jul 27, 2011 at 2:04 PM, John Chludzinski < >> jchludzinski at gmail.com> >> > wrote: >> >> >> >> Without wading through the attached zip file, it looks like you're >> missing >> >> some typedef's that come from a missing include file. ---John >> >> >> >> On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei >> wrote: >> >>> >> >>> Dear Sir/Madam, >> >>> I hope you're having a nice day. >> >>> I asked several questions about >> src/ksp/ksp/example/tutorial/ex29.c >> >>> before, and learned some beginning staff of PETSc. Right now, I want >> to >> >>> start to use it to my own program. Firstly, I want to isolate the >> ComputeRHS >> >>> function in ex29, which means I want to put ComputeRHS to another c >> program >> >>> file. I did that; however, some problem pop up when I compile it. >> >>> >> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >> >>> ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >> -Wno-unknown-pragmas >> >>> -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >> >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >> >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >> >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c >> >>> ex29.c: In function ComputeRHS: >> >>> ex29.c:39: error: storage class specified for parameter BCType >> >>> ex29.c:44: error: expected specifier-qualifier-list before BCType >> >>> ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { >> token >> >>> ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { >> token >> >>> ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { >> token >> >>> ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { >> token >> >>> ex29.c:220: error: old-style parameter declarations in prototyped >> >>> function definition >> >>> ex29.c:220: error: expected { at end of input >> >>> make: [ex29.o] Error 1 (ignored) >> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >> >>> ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >> >>> -Wno-unknown-pragmas -g3 >> -I/home/zlwei/soft/mercurial/petsc-dev/include >> >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >> >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >> >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c >> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc >> -Wall >> >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o ex29 >> >>> ex29.o ComputeRHS.o >> >>> -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >> -lpetsc >> >>> >> -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >> >>> -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl >> -lmpich >> >>> -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl >> >>> -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl >> >>> gcc: ex29.o: No such file or directory >> >>> make: [ex29] Error 1 (ignored) >> >>> >> >>> The program and modified makefile are attached. Could you please take >> a >> >>> look and give me some suggestions. >> >>> >> >>> best, >> >>> Alan >> >> >> > >> > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Thu Jul 28 13:55:56 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 28 Jul 2011 11:55:56 -0700 Subject: [petsc-users] memory corruption Message-ID: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> Hello all, I tried to use superlu as a direct solver running on Hopper, but found that there are some memory corruption errors: x/xyuan> cd $PBS_O_WORKDIR Directory: /global/homes/x/xyuan/Workspace_Nersc/cartmhdpdslin/trunk/test_superlu_as_direct_solver/m256_p1024 test_superlu_as_direct_solver/m256_p1024> aprun -n 1024 ./twcartffxmhd.exe -options_file option_twcartffxmhd_256 [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25137 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25146 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25144 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25133 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25136 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25142 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25145 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25148 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25149 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25147 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25135 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25134 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25138 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25141 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25140 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25139 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25132 : Permission denied [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25143 : Permission denied ********************************************* cartesian coordinate code(np = 1024) start time = 0.0000 time accuracy order = 2 viscosity = 0.0500 resistivity = 0.0050 skin depth = 1.0000 hyper resistivity = 0.00000630 hyper viscosity = 0.00503929 problem size: 256 by 256 dt = 0.1000 ********************************************* ******* start solving for time = 0.10000 at time step = 1****** 0 SNES Function norm 6.220836330249e-03 Linear solve converged due to CONVERGED_ITS iterations 1 1 SNES Function norm 3.041982522542e-07 *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe*** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: malloc(): memory corruption: 0x0000000001e31280 *** Any idea what is wrong here? Thanks very much! Xuefei (Rebecca) Yuan Postdoctoral Fellow Lawrence Berkeley National Laboratory Tel: 1-510-486-7031 From abhyshr at mcs.anl.gov Thu Jul 28 14:03:32 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Thu, 28 Jul 2011 14:03:32 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: <407CFB56-B9B1-4F21-8680-8E20049C072E@mcmillan-mcgee.com> Message-ID: <851979467.170977.1311879812213.JavaMail.root@zimbra.anl.gov> ----- Original Message ----- > Hi Barry, Shri, > > I reworked my implementation to include only the time-independent > electrical problem, and to make use of SNESVISetVariableBounds(). > However, it does not work quite as I expected: Constraints on > variables that are functions of other variables have no effect on the > variables by which the functions are defined. > > My constraints are set on the four degrees of freedom as follows: > -SNES_VI_INF <= V_block <= SNES_VI_INF > -SNES_VI_INF <= V_electrode <= SNES_VI_INF > 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) > 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) Note that while V_block and V_electrode and complex quantities,the magnitude terms are actually real numbers. What function did you use to compute the magnitude? abs()? sqrt of product of complex number with its conjugate? This seems like an application where the variables are of type complex but the constraints need to be set for functions of the variables which are actually real numbers. I am not sure whether this can be supported. Maybe Barry or others can shed more light on this. > > The residuals are set as follows, and the Jacobian entries are set > correspondingly: > f[V_block] = {function of adjacent V_blocks, V_electrode} > f[V_electrode] = 0.0 + 0.0i > f[magnitude(V_electrode)] = 0.0 + 0.0i > f[magnitude(I_electrode)] = 0.0 + 0.0i What equations do you use for the partial derivatives of the magnitude functions w.r.t. the actual complex variables (V_block and V_electrode) in the jacobian evaluation? > > The first two potentials, V_block and V_electrode, are independent, > while magnitude(V_electrode) is a function of V_electrode only and the > electrode current magnitude(I_electrode) is a function of V_block and > V_electrode. Note that V_electrode acts as a source term in the finite > difference formulation for V_block. While all of these variables are > complex, the latter two always have zero imaginary parts. I suppose I > am assuming that PETSc knows to compare only the real parts if the > imaginary parts are zero. (Is this a bad assumption?) > > When solving with PETSc (using -snes_vi_type rs) Did you also use -snes_type vi? V_electrode stays at > its maximum, as set in the initial guess, and the V_block distribution > falls properly from that. However, measurements of > magnitude(I_electrode) are way above maximum and do not move downwards > with successive SNES iterations. I can also set the constraint on > magnitude(V_electrode) to less than maximum and it does not affect the > value of V_electrode. How can I tell PETSc to change V_electrode when > the magnitude(V_electrode) or magnitude(I_electrode) constraints are > not met? > Send the code for your application with instructions on how to run it and we'll try to figure out what needs to be done. Shri > Thanks again for your help. > > Jon > > > On 2011-07-25, at 8:58 PM, Barry Smith wrote: > > > > > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > > > >> Hi Shri, > >> > >> Thanks for your message and all the helpful tips. If the > >> TSVISetVariableBounds() functions are available now, I would like > >> to try them as well. Is the interface essentially the same as with > >> SNESVISetVariableBounds()? I will get back to you and Barry when I > >> have had a chance to modify my application. > >> > >> For my problem, I believe it makes sense to have bounds on one of > >> the variables as well as one function of the variables. The two > >> relevant degrees of freedom are the block voltage (one for each > >> finite difference block) and the electrode voltage (one for each > >> electrode, which may be present in multiple blocks). The electrode > >> voltage should keep a constant phase while the magnitude is > >> constrained between zero and some maximum. The block voltages near > >> the electrodes depend on the electrode voltages as well as the > >> neighbouring block voltages. The electrode current for a given > >> electrode is a function of its electrode voltage and several nearby > >> block voltages, and should be constrained in magnitude between zero > >> and some maximum (and the phase unconstrained). Would I need to use > >> SNESVISetComputeVariableBounds since the electrode current is a > >> function of the other variables? Would any other provisions need to > >> be made for the block voltages, since they depend on the electrode > >> voltages? > >> > > > > You cannot directly make a bound on some function of other > > variables. Instead you need to introduce another variable that is > > defined to be equal to that function and put the bound on that new > > variable. > > > > Barry > > > >> Thank you again, > >> > >> Jon > >> > >> On 2011-07-25, at 11:58 AM, Shri wrote: > >> > >>> > >>> ----- Original Message ----- > >>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > >>>> > >>>>> Barry, > >>>>> > >>>>> Thank you so much for your response. Lucky, indeed! I look > >>>>> forward > >>>>> to trying out these new features. > >>>>> > >>>>> I neglected to mention in my original post that my electrical > >>>>> problem is part of a DAE, which includes a time-dependent > >>>>> heating > >>>>> problem. Can SNESVI constraints be used in conjunction with > >>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I > >>>>> only > >>>>> need the constraints for the time-independent electrical > >>>>> portion.) > >>>> > >>>> We have not yet put that in but Shri is starting to look at that > >>>> just > >>>> now. Basically we would have a TSVISetVariableBounds() and handle > >>>> everything from there. I suggest you start with a simple time > >>>> electrical portion with constraints to explore and we'll go from > >>>> there. Shri is actually a electrical networks guy and so can > >>>> speak > >>>> your language. > >>> > >>> > >>> I've added TSVISetVariableBounds() for setting the bounds on the > >>> variables using the TS object directly. > >>> A few things that i want to mention here about using the > >>> variational inequality nonlinear solver (SNESVI). > >>> i) Use the runtime option -snes_type vi or explicitly set > >>> SNESSetType(snes,SNESVI). > >>> ii) There are two tested algorithms currently available, (a) > >>> semismooth (-snes_vi_type ss) and (b) active set or reduced space > >>> (-snes_vi_type rs). > >>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials > >>> which is a time-stepping nonlinear problem with constraints on the > >>> variables. This example does not use the TS object directly but > >>> rather a time-loop is explicitly written. Another example,ex8.c, > >>> in src/snes/examples/tests/ is a minimum surface area problem > >>> which uses SNESVI. > >>> > >>> By the way, does your problem have bounds on the variables or > >>> bounds on some function of the variables? > >>> > >>> Shri > >>> > >>> > >>> > >>>> > >>>> Barry > >>>> > >>>> > >>>> > >>>>> > >>>>> Thank you again, > >>>>> > >>>>> Jon > >>>>> > >>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: > >>>>> > >>>>>> > >>>>>> Jon, > >>>>>> > >>>>>> You may be in luck. In PETSc-dev > >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we > >>>>>> have now implemented variational inequality non-linear solvers > >>>>>> with box constraints. > >>>>>> > >>>>>> That is one has a set of variables u and algebraic equations > >>>>>> F(u) = 0 (say of size n each) but in addition one has > >>>>>> constraints lu <= u <= uu where some or all of lu may be > >>>>>> negative infinity (no constraints) and some or all of uu may be > >>>>>> infinity (no constraints). There is also a constraint on the > >>>>>> sign of F() for those equations associated with active > >>>>>> constraints. If your constraints are not in this form sometimes > >>>>>> you can introduce new additional variables to get it in this > >>>>>> form. Read up a little on variational inequalities on the web. > >>>>>> > >>>>>> To use this you provide the usual SNES function and Jacobian > >>>>>> but > >>>>>> you also provide SNESVISetVariableBounds() there is a manual > >>>>>> page for this function and for for SNESVI at > >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html > >>>>>> (the link is broken right now but Satish is fixing it). Plus > >>>>>> several examples in src/snes/examples/tutorials (in petsc-dev). > >>>>>> > >>>>>> This is all new code so we would be interested in your > >>>>>> feedback. > >>>>>> > >>>>>> > >>>>>> Barry > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I am trying to add a constraint feature to my first PETSc > >>>>>>> application, which uses the finite difference method to > >>>>>>> calculate > >>>>>>> the potential distribution produced by a collection of > >>>>>>> electrodes > >>>>>>> in a resistive medium. I would like to make this simulation > >>>>>>> more > >>>>>>> realistic by imposing a maximum electric current and a maximum > >>>>>>> potential difference that can be supplied to each electrode by > >>>>>>> the > >>>>>>> power supply. If the medium between the electrodes is very > >>>>>>> conductive, the current maximum would be exceeded by the > >>>>>>> maximum > >>>>>>> potential difference, so the potential difference should be > >>>>>>> decreased from maximum until it produces the maximum current. > >>>>>>> On > >>>>>>> the other hand, the potential difference between the > >>>>>>> electrodes > >>>>>>> should remain at maximum as long as the current remains below > >>>>>>> maximum (say, for a less conductive medium). > >>>>>>> > >>>>>>> I added an extra degree of freedom (the electrode voltages) to > >>>>>>> my > >>>>>>> DMDA, and I developed a set of conditional expressions that > >>>>>>> describe the above constraints, but one problem is that the > >>>>>>> logic > >>>>>>> relies on if-then-else decisions that are made when forming > >>>>>>> the > >>>>>>> function/residual and the Jacobian. Once these decisions are > >>>>>>> made, > >>>>>>> of course, the conditions are not checked again until the next > >>>>>>> function or Jacobian evaluation. The non-linear solver then > >>>>>>> tends > >>>>>>> to oscillate between extreme solutions to the opposing > >>>>>>> conditions > >>>>>>> with each iteration, and never converges towards a reasonable > >>>>>>> solution. > >>>>>>> > >>>>>>> Is there a better strategy for solving such problems? Does > >>>>>>> PETSc > >>>>>>> offer mechanisms to aid in their solution? I would very much > >>>>>>> appreciate any hints. > >>>>>>> > >>>>>>> Thank you for your time, > >>>>>>> > >>>>>>> Jon > >>>>>> > >>>>> > >>> > >> > > From bsmith at mcs.anl.gov Thu Jul 28 14:26:47 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 28 Jul 2011 14:26:47 -0500 Subject: [petsc-users] Retrieving Individual Values from a Parallel Matrix In-Reply-To: References: Message-ID: On Jul 28, 2011, at 12:44 PM, Adam Byrd wrote: > Dear All, > > I'm attempting to find the most efficient/simple way to collectively retrieve just a handful of values (local and non local) from a parallel matrix. I'm filling two 4x4 arrays with 4 values from 8 different rows of the solution matrix of a previous solve. So far, MatGetSubMatrices looks my only option (MatGetValues and MatGetRow can only get local values), but seems unlikely to be the most efficient way to do this unless I am able to change the row and column ordering when creating the submatrix. It appears to retain the relative ordering of the rows and columns from the original matrix. Yes it does keep the same ordering from the original matrix. You need to do the operation in two steps. Step 1 use MatGetSubMatrices() or MatGetSubMatrix() to get onto each MPI process the rows/columns that it will need. Step 2 go through the (now local) part of the matrix and get the values you want out and put them where you want using the ordering that you want. Barry > Essentially, I'm trying to do this: > > myArray[n][m] = myPetscMat[i][j] > > This is the code I'm rewriting to work collectively on a parallel matrix: > > for(int i=0; i<4; i++) > { > ierr = MatGetRow(inverseHamiltonian, isl[i]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); > for(int j=0; j<4; j++) > { > gaml[i][j] = rowValues[isr[j]-1]; > } //endfor > ierr = MatRestoreRow(inverseHamiltonian, isl[i]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); > } //endfor > > for(int j=0; j<4; j++) > { > ierr = MatGetRow(inverseHamiltonian, isr[j]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); > for(int i=0; i<4; i++) > { > gamrr[i][j] = conj(rowValues[isl[i]-1]); > } //endfor > ierr = MatRestoreRow(inverseHamiltonian, isr[j]-1, &nonZeros, PETSC_NULL, &rowValues);CHKERRQ(ierr); > } //endfor > > where isr[] and isl[] contain arbitrary, unsorted indices. > > Thanks. From hzhang at mcs.anl.gov Thu Jul 28 15:05:24 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 28 Jul 2011 15:05:24 -0500 Subject: [petsc-users] memory corruption In-Reply-To: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> Message-ID: Rebecca: Turn off orderings and some options, e.g., -mat_superlu_dist_equil NO -mat_superlu_dist_rowperm NATURAL -mat_superlu_dist_colperm NATURAL Do you still get memory corruption? Hong > Hello all, > > I tried to use superlu as a direct solver running on Hopper, but found that there are some memory corruption errors: > > x/xyuan> cd $PBS_O_WORKDIR > Directory: /global/homes/x/xyuan/Workspace_Nersc/cartmhdpdslin/trunk/test_superlu_as_direct_solver/m256_p1024 > test_superlu_as_direct_solver/m256_p1024> aprun -n 1024 ./twcartffxmhd.exe -options_file option_twcartffxmhd_256 > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25137 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25146 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25144 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25133 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25136 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25142 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25145 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25148 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25149 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25147 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25135 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25134 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25138 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25141 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25140 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25139 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25132 : Permission denied > [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25143 : Permission denied > ********************************************* > cartesian coordinate code(np = 1024) > start time = 0.0000 > time accuracy order = 2 > viscosity = 0.0500 > resistivity = 0.0050 > skin depth = 1.0000 > hyper resistivity = 0.00000630 > hyper viscosity = 0.00503929 > problem size: 256 by 256 > dt = 0.1000 > ********************************************* > ******* start solving for time = 0.10000 at time step = 1****** > ?0 SNES Function norm 6.220836330249e-03 > Linear solve converged due to CONVERGED_ITS iterations 1 > ?1 SNES Function norm 3.041982522542e-07 > *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe*** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: malloc(): memory corruption: 0x0000000001e31280 *** > > Any idea what is wrong here? > > Thanks very much! > > Xuefei (Rebecca) Yuan > Postdoctoral Fellow > Lawrence Berkeley National Laboratory > Tel: 1-510-486-7031 > > > From jbacks at mcmillan-mcgee.com Thu Jul 28 17:04:42 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Thu, 28 Jul 2011 16:04:42 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <851979467.170977.1311879812213.JavaMail.root@zimbra.anl.gov> References: <851979467.170977.1311879812213.JavaMail.root@zimbra.anl.gov> Message-ID: <78FAC6C2-F55B-4DA1-B14E-0EEC9E4CF0AF@mcmillan-mcgee.com> Thanks for your reply, Shri. Responses below: On 2011-07-28, at 1:03 PM, Shri wrote: > > ----- Original Message ----- >> Hi Barry, Shri, >> >> I reworked my implementation to include only the time-independent >> electrical problem, and to make use of SNESVISetVariableBounds(). >> However, it does not work quite as I expected: Constraints on >> variables that are functions of other variables have no effect on the >> variables by which the functions are defined. >> >> My constraints are set on the four degrees of freedom as follows: >> -SNES_VI_INF <= V_block <= SNES_VI_INF >> -SNES_VI_INF <= V_electrode <= SNES_VI_INF >> 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) >> 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) > > Note that while V_block and V_electrode and complex quantities,the magnitude terms are actually real numbers. What function did you use to compute the magnitude? abs()? sqrt of product of complex number with its conjugate? I used the "0.0 + 0.0i" there to emphasize that I built PETSc with complex scalars, so all the scalars are stored as complex even if the imaginary components are always zero. I have been using std::abs() from the C++ standard library complex template class (std::complex). I believe the values of type 'double' returned by std::abs() are implicitly cast to the real part of std::complex or PetscScalar, though I usually try to cast explicitly. > This seems like an application where the variables are of type complex but the constraints need to be set for functions of the variables which are actually real numbers. I am not sure whether this can be supported. Maybe Barry or others can shed more > light on this. My understanding is that comparison operators (such as < and >) are not defined for complex numbers, so would it be correct to say that anyone using complex scalars within PETSc will encounter this problem with SNESVI? In other words, SNESVI inequality constraints can only be applied to real variables or real-valued functions of complex variables? >> >> The residuals are set as follows, and the Jacobian entries are set >> correspondingly: >> f[V_block] = {function of adjacent V_blocks, V_electrode} >> f[V_electrode] = 0.0 + 0.0i >> f[magnitude(V_electrode)] = 0.0 + 0.0i >> f[magnitude(I_electrode)] = 0.0 + 0.0i > > What equations do you use for the partial derivatives of the magnitude functions w.r.t. the actual complex variables (V_block and V_electrode) in the jacobian evaluation? Since I set all the residuals for both magnitude variables to zero, and each entry in the Jacobian matrix is a partial derivative of a residual, I believe the Jacobian entries for both magnitudes would all be zero as well. However, your point made me realize I have no Jacobian entry for the partial derivative of f[V_block] w.r.t. V_electrode. Is that needed? How would one enter it with MatSetValuesStencil? My working understanding was that each degree of freedom conceptually has its own residual and Jacobian, even though all residuals are contained in one Vec and all Jacobians contained in one Mat. >> >> The first two potentials, V_block and V_electrode, are independent, >> while magnitude(V_electrode) is a function of V_electrode only and the >> electrode current magnitude(I_electrode) is a function of V_block and >> V_electrode. Note that V_electrode acts as a source term in the finite >> difference formulation for V_block. While all of these variables are >> complex, the latter two always have zero imaginary parts. I suppose I >> am assuming that PETSc knows to compare only the real parts if the >> imaginary parts are zero. (Is this a bad assumption?) >> >> When solving with PETSc (using -snes_vi_type rs) > > Did you also use -snes_type vi? I have been using SNESSetType(snes, SNESVI) in my code, and the use of -snes_type vi in the command line does not seem to change any behaviour. > > V_electrode stays at >> its maximum, as set in the initial guess, and the V_block distribution >> falls properly from that. However, measurements of >> magnitude(I_electrode) are way above maximum and do not move downwards >> with successive SNES iterations. I can also set the constraint on >> magnitude(V_electrode) to less than maximum and it does not affect the >> value of V_electrode. How can I tell PETSc to change V_electrode when >> the magnitude(V_electrode) or magnitude(I_electrode) constraints are >> not met? >> > Send the code for your application with instructions on how to run it and we'll try to figure out what needs to be > done. I really appreciate this generous offer of your time, and I will certainly take you up on it if we cannot resolve this otherwise. Unfortunately, I will be travelling for the next few weeks and it may take me some time to isolate the PETSc-dependent parts of my application. Please bear with me as I will try to keep in touch while I am away. As I mentioned above, I think the issue may lie in the conceptual incompatibility of complex variables and inequality constraints, but I would appreciate your thoughts on that. Thanks again, Jon > > > Shri > > >> Thanks again for your help. >> >> Jon >> >> >> On 2011-07-25, at 8:58 PM, Barry Smith wrote: >> >>> >>> On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: >>> >>>> Hi Shri, >>>> >>>> Thanks for your message and all the helpful tips. If the >>>> TSVISetVariableBounds() functions are available now, I would like >>>> to try them as well. Is the interface essentially the same as with >>>> SNESVISetVariableBounds()? I will get back to you and Barry when I >>>> have had a chance to modify my application. >>>> >>>> For my problem, I believe it makes sense to have bounds on one of >>>> the variables as well as one function of the variables. The two >>>> relevant degrees of freedom are the block voltage (one for each >>>> finite difference block) and the electrode voltage (one for each >>>> electrode, which may be present in multiple blocks). The electrode >>>> voltage should keep a constant phase while the magnitude is >>>> constrained between zero and some maximum. The block voltages near >>>> the electrodes depend on the electrode voltages as well as the >>>> neighbouring block voltages. The electrode current for a given >>>> electrode is a function of its electrode voltage and several nearby >>>> block voltages, and should be constrained in magnitude between zero >>>> and some maximum (and the phase unconstrained). Would I need to use >>>> SNESVISetComputeVariableBounds since the electrode current is a >>>> function of the other variables? Would any other provisions need to >>>> be made for the block voltages, since they depend on the electrode >>>> voltages? >>>> >>> >>> You cannot directly make a bound on some function of other >>> variables. Instead you need to introduce another variable that is >>> defined to be equal to that function and put the bound on that new >>> variable. >>> >>> Barry >>> >>>> Thank you again, >>>> >>>> Jon >>>> >>>> On 2011-07-25, at 11:58 AM, Shri wrote: >>>> >>>>> >>>>> ----- Original Message ----- >>>>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: >>>>>> >>>>>>> Barry, >>>>>>> >>>>>>> Thank you so much for your response. Lucky, indeed! I look >>>>>>> forward >>>>>>> to trying out these new features. >>>>>>> >>>>>>> I neglected to mention in my original post that my electrical >>>>>>> problem is part of a DAE, which includes a time-dependent >>>>>>> heating >>>>>>> problem. Can SNESVI constraints be used in conjunction with >>>>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I >>>>>>> only >>>>>>> need the constraints for the time-independent electrical >>>>>>> portion.) >>>>>> >>>>>> We have not yet put that in but Shri is starting to look at that >>>>>> just >>>>>> now. Basically we would have a TSVISetVariableBounds() and handle >>>>>> everything from there. I suggest you start with a simple time >>>>>> electrical portion with constraints to explore and we'll go from >>>>>> there. Shri is actually a electrical networks guy and so can >>>>>> speak >>>>>> your language. >>>>> >>>>> >>>>> I've added TSVISetVariableBounds() for setting the bounds on the >>>>> variables using the TS object directly. >>>>> A few things that i want to mention here about using the >>>>> variational inequality nonlinear solver (SNESVI). >>>>> i) Use the runtime option -snes_type vi or explicitly set >>>>> SNESSetType(snes,SNESVI). >>>>> ii) There are two tested algorithms currently available, (a) >>>>> semismooth (-snes_vi_type ss) and (b) active set or reduced space >>>>> (-snes_vi_type rs). >>>>> iii) Take a look at example,ex61.c, in src/snes/examples/tutorials >>>>> which is a time-stepping nonlinear problem with constraints on the >>>>> variables. This example does not use the TS object directly but >>>>> rather a time-loop is explicitly written. Another example,ex8.c, >>>>> in src/snes/examples/tests/ is a minimum surface area problem >>>>> which uses SNESVI. >>>>> >>>>> By the way, does your problem have bounds on the variables or >>>>> bounds on some function of the variables? >>>>> >>>>> Shri >>>>> >>>>> >>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thank you again, >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: >>>>>>> >>>>>>>> >>>>>>>> Jon, >>>>>>>> >>>>>>>> You may be in luck. In PETSc-dev >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we >>>>>>>> have now implemented variational inequality non-linear solvers >>>>>>>> with box constraints. >>>>>>>> >>>>>>>> That is one has a set of variables u and algebraic equations >>>>>>>> F(u) = 0 (say of size n each) but in addition one has >>>>>>>> constraints lu <= u <= uu where some or all of lu may be >>>>>>>> negative infinity (no constraints) and some or all of uu may be >>>>>>>> infinity (no constraints). There is also a constraint on the >>>>>>>> sign of F() for those equations associated with active >>>>>>>> constraints. If your constraints are not in this form sometimes >>>>>>>> you can introduce new additional variables to get it in this >>>>>>>> form. Read up a little on variational inequalities on the web. >>>>>>>> >>>>>>>> To use this you provide the usual SNES function and Jacobian >>>>>>>> but >>>>>>>> you also provide SNESVISetVariableBounds() there is a manual >>>>>>>> page for this function and for for SNESVI at >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html >>>>>>>> (the link is broken right now but Satish is fixing it). Plus >>>>>>>> several examples in src/snes/examples/tutorials (in petsc-dev). >>>>>>>> >>>>>>>> This is all new code so we would be interested in your >>>>>>>> feedback. >>>>>>>> >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I am trying to add a constraint feature to my first PETSc >>>>>>>>> application, which uses the finite difference method to >>>>>>>>> calculate >>>>>>>>> the potential distribution produced by a collection of >>>>>>>>> electrodes >>>>>>>>> in a resistive medium. I would like to make this simulation >>>>>>>>> more >>>>>>>>> realistic by imposing a maximum electric current and a maximum >>>>>>>>> potential difference that can be supplied to each electrode by >>>>>>>>> the >>>>>>>>> power supply. If the medium between the electrodes is very >>>>>>>>> conductive, the current maximum would be exceeded by the >>>>>>>>> maximum >>>>>>>>> potential difference, so the potential difference should be >>>>>>>>> decreased from maximum until it produces the maximum current. >>>>>>>>> On >>>>>>>>> the other hand, the potential difference between the >>>>>>>>> electrodes >>>>>>>>> should remain at maximum as long as the current remains below >>>>>>>>> maximum (say, for a less conductive medium). >>>>>>>>> >>>>>>>>> I added an extra degree of freedom (the electrode voltages) to >>>>>>>>> my >>>>>>>>> DMDA, and I developed a set of conditional expressions that >>>>>>>>> describe the above constraints, but one problem is that the >>>>>>>>> logic >>>>>>>>> relies on if-then-else decisions that are made when forming >>>>>>>>> the >>>>>>>>> function/residual and the Jacobian. Once these decisions are >>>>>>>>> made, >>>>>>>>> of course, the conditions are not checked again until the next >>>>>>>>> function or Jacobian evaluation. The non-linear solver then >>>>>>>>> tends >>>>>>>>> to oscillate between extreme solutions to the opposing >>>>>>>>> conditions >>>>>>>>> with each iteration, and never converges towards a reasonable >>>>>>>>> solution. >>>>>>>>> >>>>>>>>> Is there a better strategy for solving such problems? Does >>>>>>>>> PETSc >>>>>>>>> offer mechanisms to aid in their solution? I would very much >>>>>>>>> appreciate any hints. >>>>>>>>> >>>>>>>>> Thank you for your time, >>>>>>>>> >>>>>>>>> Jon >>>>>>>> >>>>>>> >>>>> >>>> >>> > From xyuan at lbl.gov Thu Jul 28 17:06:45 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 28 Jul 2011 15:06:45 -0700 Subject: [petsc-users] memory corruption In-Reply-To: References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> Message-ID: Dear Hong, Yes, there is no memory corruption for small size problem test after I add those options. I have not tested on big size problems yet. For using superlu_dist as a direct solver, I would like to make sure that the there is no memory corruption for large size problem, i.e., the memory requirement for a single processor does not exceed the memory of each processor. For example, on Hopper, 1 node has 24 cores, and if I have 24 cores per node, each core has the memory approximately 1.33 Gb. Assume the problem size is M*N, and there are 4 unknowns per grid point, and the standard 13-point scheme is used for discretization. Therefore, the number of non zeroes per row is 13. The sparse matrix is 4MN X 4MN in size with 13*4MN non zeroes. Assume that the data type is PetscScalar (8 bytes), what would be the memory usage for the matrix? I ran a test problem of 64X64 for 10 time steps in np=1,2,4,8,16,32. With the help of '-log_summary', I was able to tell the memory usage, for example, when np=32, Matrix 9 9 1172968 0 The memory usage for processor 0 is 1,172,968 bytes in Matrix. Is this 1,172,968 bytes the total number for those 9 creations/destructions, or how could I understand this 1,172,968 come from? I did the math as follow, the non zeroes of the matrix is 13*4MN=212,992; and the memory is 8(bytes for PetscScalar) * 212,992 = 1,703,936; on each processor, the memory is 1,703,936/32 = 53,248 this 53,248 is far from 1,172,968. even if I multiply by 10 (for 10 time steps iterations), 530,248 is still far from 1,172,958. Where did I get this number wrong? -log_summary returns: np=1 Matrix 3 3 38785428 0 np=2 Matrix 9 9 19871976 0 np=4 Matrix 9 9 9948136 0 np=8 Matrix 9 9 4894696 0 np=16 Matrix 9 9 2413544 0 np=32 Matrix 9 9 1172968 0 for other cases. Thanks very much! Best regards, Rebecca On Jul 28, 2011, at 1:05 PM, Hong Zhang wrote: > Rebecca: > > Turn off orderings and some options, e.g., > -mat_superlu_dist_equil NO -mat_superlu_dist_rowperm NATURAL > -mat_superlu_dist_colperm NATURAL > > Do you still get memory corruption? > > Hong > >> Hello all, >> >> I tried to use superlu as a direct solver running on Hopper, but found that there are some memory corruption errors: >> >> x/xyuan> cd $PBS_O_WORKDIR >> Directory: /global/homes/x/xyuan/Workspace_Nersc/cartmhdpdslin/trunk/test_superlu_as_direct_solver/m256_p1024 >> test_superlu_as_direct_solver/m256_p1024> aprun -n 1024 ./twcartffxmhd.exe -options_file option_twcartffxmhd_256 >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25137 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25146 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25144 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25133 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25136 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25142 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25145 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25148 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25149 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25147 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25135 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25134 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25138 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25141 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25140 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25139 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25132 : Permission denied >> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25143 : Permission denied >> ********************************************* >> cartesian coordinate code(np = 1024) >> start time = 0.0000 >> time accuracy order = 2 >> viscosity = 0.0500 >> resistivity = 0.0050 >> skin depth = 1.0000 >> hyper resistivity = 0.00000630 >> hyper viscosity = 0.00503929 >> problem size: 256 by 256 >> dt = 0.1000 >> ********************************************* >> ******* start solving for time = 0.10000 at time step = 1****** >> 0 SNES Function norm 6.220836330249e-03 >> Linear solve converged due to CONVERGED_ITS iterations 1 >> 1 SNES Function norm 3.041982522542e-07 >> *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe*** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: malloc(): memory corruption: 0x0000000001e31280 *** >> >> Any idea what is wrong here? >> >> Thanks very much! >> >> Xuefei (Rebecca) Yuan >> Postdoctoral Fellow >> Lawrence Berkeley National Laboratory >> Tel: 1-510-486-7031 >> >> >> From mirzadeh at gmail.com Thu Jul 28 22:49:21 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Thu, 28 Jul 2011 20:49:21 -0700 Subject: [petsc-users] Grid Partitioning with ParMetis Message-ID: Hi all, I am trying to write a code to do parallel computation on quadtree adaptive grids and to do so , I need to distribute the grid in parallel. I have selected a general unstructured framework for telling PETSc about my node numbering. An example of such grid is schematically shown below. 1 16 7 3 +---------------+---------------+------------------------------+ | | | | | | | | |14 | 15 | 17 | +---------------+---------------+ | | | | | | | | | | 4 | 12 | 6 |8 +---------------+---------------+------------------------------+ | | | | | | | | | 9 | 11 | 13 | +---------------+---------------+ | | | | | | | | | | 0 | 10 |5 | 2 +---------------+---------------+------------------------------+ To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: Number of elements in ordering 18 PETSc->App App->PETSc 0 9 0 1 1 0 1 3 2 10 2 4 3 1 3 7 4 2 4 12 5 11 5 14 6 12 6 15 7 3 7 16 8 13 8 17 9 14 9 0 10 15 10 2 11 16 11 5 12 4 12 6 13 17 13 8 14 5 14 9 15 6 15 10 16 7 16 11 17 8 17 13 Now I have two questions/concerns: 1) Do processors always have the nodes in contiguous chunks of PETSc ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular ordering does not seem to be "good" for this grid since it seems to cross too many edges in the graph (here 13 edges) and by just looking at the graph I can(at least) think of a better distribution with only 6 edge cuts. (if you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on rank 1). 2) Isn't it true that the final distribution should be independent of initial grid numbering? When I try the same grid but with the following (hand-generated) numbering: 14 15 16 17 +---------------+---------------+------------------------------+ | | | | | | | | |11 | 12 | 13 | +---------------+---------------+ | | | | | | | | | | 7 | 8 | 9 |10 +---------------+---------------+------------------------------+ | | | | | | | | | 4 | 5 | 6 | +---------------+---------------+ | | | | | | | | | | 0 | 1 |2 | 3 +---------------+---------------+------------------------------+ I get the following AO: Number of elements in ordering 18 PETSc->App App->PETSc 0 9 0 9 1 10 1 10 2 11 2 11 3 12 3 12 4 13 4 13 5 14 5 14 6 15 6 15 7 16 7 16 8 17 8 17 9 0 9 0 10 1 10 1 11 2 11 2 12 3 12 3 13 4 13 4 14 5 14 5 15 6 15 6 16 7 16 7 17 8 17 8 which is simply the initial ordering with a change in the order in which processors handle nodes. Could it be that the partitioning is not unique and each time the algorithm only tries to obtain the "best" possible ordering depending on the initial distribution? If so, how should I know what ordering to start with? I am really confused and would appreciate if someone could provide some insights. Thanks, Mohammad -------------- next part -------------- An HTML attachment was scrubbed... URL: From Debao.Shao at brion.com Thu Jul 28 22:57:39 2011 From: Debao.Shao at brion.com (Debao Shao) Date: Thu, 28 Jul 2011 20:57:39 -0700 Subject: [petsc-users] (no subject) In-Reply-To: References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> Message-ID: <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> Dear all: I'm using PETSC iterative solver(PCILU & KSPGMRES) in OOQP, but when the to be solved matrix is ill-posed, say condition number ~ 1e+4, then, KSPGMRES will exceed its maximal iteration number(default as 10000), while when I checked the same data with cholmod direct solver, it can solve the problem and return an answer correctly. Does it mean the iterative solver(like GMRES) relies more on the well-pose matrix than cholmod direct solver? How can I improve the case to avoid KSPGMRES fail case? Your answer is appreciated. Thanks, Debao -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. From Debao.Shao at brion.com Thu Jul 28 23:00:40 2011 From: Debao.Shao at brion.com (Debao Shao) Date: Thu, 28 Jul 2011 21:00:40 -0700 Subject: [petsc-users] the convergence of petsc iterative solver In-Reply-To: <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> Message-ID: <384FF55F15E3E447802DC8CCA85696980E13EB8C94@EX03> Dear all: I'm using PETSC iterative solver(PCILU & KSPGMRES) in OOQP, but when the to be solved matrix is ill-posed, say condition number ~ 1e+4, then, KSPGMRES will exceed its maximal iteration number(default as 10000), while when I checked the same data with cholmod direct solver, it can solve the problem and return an answer correctly. Does it mean the iterative solver(like GMRES) relies more on the well-pose matrix than cholmod direct solver? How can I improve the case to avoid KSPGMRES fail case? Your answer is appreciated. Thanks, Debao -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. From bsmith at mcs.anl.gov Thu Jul 28 23:11:04 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 28 Jul 2011 23:11:04 -0500 Subject: [petsc-users] (no subject) In-Reply-To: <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> Message-ID: <715C3B78-1781-4DE4-94F0-8289C4068138@mcs.anl.gov> On Jul 28, 2011, at 10:57 PM, Debao Shao wrote: > Dear all: > > I'm using PETSC iterative solver(PCILU & KSPGMRES) in OOQP, but when the to be solved matrix is ill-posed, say condition number ~ 1e+4, then, KSPGMRES will exceed its maximal iteration number(default as 10000), while when I checked the same data with cholmod direct solver, it can solve the problem and return an answer correctly. > > Does it mean the iterative solver(like GMRES) relies more on the well-pose matrix than cholmod direct solver? Yes, generally iterative solvers take a longer time for more ill-conditioned matrices. Direct solvers are not usually effected much by ill conditioning until it becomes large. > How can I improve the case to avoid KSPGMRES fail case? There are many many choices in PETSc making it hard to know how to get started on getting an efficient solver. 1) When the matrix is very ill-conditioned or small (say less than 1000 by 1000) then the best bet is a direct solver -pc_type lu (and in parallel you need to install an external solver package MUMPS or SuperLU_dist see http://www.mcs.anl.gov/petsc/petsc-as/documentation/linearsolvertable.html) 2) in your case condition number of ~ 10,000 is low enough that iterative solver can likely be faster than direct solver so it is worth investigating iterative solvers. The first thing to ask is there any special structure in the matrix you can take advantage of? Is is symmetric, positive definite then use -ksp_type cg Does the matrix arise from an elliptic PDE discretization on a mesh then likely you can get multigrid to work well either geometric or algebraic see http://www.mcs.anl.gov/petsc/petsc-as/documentation/linearsolvertable.html) . Does the matrix come from a fluids problem? Then there are various tricks we can help you with. If you know of no particular structure then the fall back is ILU. Since ILU(0) doesn't seem to work you can try ILU(k) for small k integer to see if that helps, just add the option -pc_factor_levels 1 or 2 or 3 and see how that affects the convergence. Always run with -ksp_monitor_true_residual -ksp_converged_reason when testing to see how the solver is working. Since the default GMRES restart is 30 that may be hurting your convergence, you can try a larger restart with -ksp_gmres_restart 100 or use -ksp_type bcgs that does not require a restart. This is enough to get you started. Are you interested in the parallel case? That introduces many more difficulties. Barry > > Your answer is appreciated. > > Thanks, > Debao > > -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. From knepley at gmail.com Thu Jul 28 23:25:04 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 29 Jul 2011 04:25:04 +0000 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh wrote: > Hi all, > > I am trying to write a code to do parallel computation on quadtree adaptive > grids and to do so , I need to distribute the grid in parallel. I have > selected a general unstructured framework for telling PETSc about my node > numbering. An example of such grid is schematically shown below. > 0) If you are doing this, I think you should at least look at the p4est package before proceeding. > > 1 16 7 3 > +---------------+---------------+------------------------------+ > | | | | > | | | | > |14 | 15 | 17 | > +---------------+---------------+ | > | | | | > | | | | > | 4 | 12 | 6 |8 > +---------------+---------------+------------------------------+ > | | | | > | | | | > | 9 | 11 | 13 | > +---------------+---------------+ | > | | | | > | | | | > | 0 | 10 |5 | 2 > +---------------+---------------+------------------------------+ > > > To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: > > > Number of elements in ordering 18 > PETSc->App App->PETSc > 0 9 0 1 > 1 0 1 3 > 2 10 2 4 > 3 1 3 7 > 4 2 4 12 > 5 11 5 14 > 6 12 6 15 > 7 3 7 16 > 8 13 8 17 > 9 14 9 0 > 10 15 10 2 > 11 16 11 5 > 12 4 12 6 > 13 17 13 8 > 14 5 14 9 > 15 6 15 10 > 16 7 16 11 > 17 8 17 13 > > Now I have two questions/concerns: > > 1) Do processors always have the nodes in contiguous chunks of PETSc > ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular > ordering does not seem to be "good" for this grid since it seems to cross > too many edges in the graph (here 13 edges) and by just looking at the graph > I can(at least) think of a better distribution with only 6 edge cuts. (if > you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on > rank 1). > Yes, the PETSc ordering is always contiguous. Perhaps you are not providing the graph you think you are for partitioning. > 2) Isn't it true that the final distribution should be independent of > initial grid numbering? When I try the same grid but with the following > (hand-generated) numbering: > > 14 15 16 17 > +---------------+---------------+------------------------------+ > | | | | > | | | | > |11 | 12 | 13 | > +---------------+---------------+ | > | | | | > | | | | > | 7 | 8 | 9 |10 > +---------------+---------------+------------------------------+ > | | | | > | | | | > | 4 | 5 | 6 | > +---------------+---------------+ | > | | | | > | | | | > | 0 | 1 |2 | 3 > +---------------+---------------+------------------------------+ > > I get the following AO: > > Number of elements in ordering 18 > PETSc->App App->PETSc > 0 9 0 9 > 1 10 1 10 > 2 11 2 11 > 3 12 3 12 > 4 13 4 13 > 5 14 5 14 > 6 15 6 15 > 7 16 7 16 > 8 17 8 17 > 9 0 9 0 > 10 1 10 1 > 11 2 11 2 > 12 3 12 3 > 13 4 13 4 > 14 5 14 5 > 15 6 15 6 > 16 7 16 7 > 17 8 17 8 > > > which is simply the initial ordering with a change in the order in which > processors handle nodes. Could it be that the partitioning is not unique > and each time the algorithm only tries to obtain the "best" possible > ordering depending on the initial distribution? If so, how should I know > what ordering to start with? > Yes, ParMetis does not provide a unique "best" ordering, which is at least NP-complete if not worse. Matt > I am really confused and would appreciate if someone could provide some > insights. > > Thanks, > Mohammad > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Thu Jul 28 23:52:33 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Thu, 28 Jul 2011 21:52:33 -0700 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est uses parallel octrees/quadtrees but for what I intend to do I only need to distribute a single tree that is created in serial among processors. I definitely like to have the tree data-structure in parallel but that would be another project. I also looked into Dendro and they kind of follow the same strategy. i.e every single processor has a local copy of the whole tree. What they do differently, however, is they somehow manage to use DA instead of a general unstructured numbering which is quite interesting but I still don't know how they do it. Unfortunately, they do not handle (as far as I understood from their manual) non-graded trees which are the ones I work with. So, all I need to do is to somehow distribute my grid among processors and since each one has a local copy of data-structure I could get around the problem. Just anotehr question. If the partitioning is not unique, do you at least get a better numbering than the tree you start with? Mohammad On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: > On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh wrote: > >> Hi all, >> >> I am trying to write a code to do parallel computation on quadtree >> adaptive grids and to do so , I need to distribute the grid in parallel. I >> have selected a general unstructured framework for telling PETSc about my >> node numbering. An example of such grid is schematically shown below. >> > > 0) If you are doing this, I think you should at least look at the p4est > package before proceeding. > > >> >> 1 16 7 3 >> +---------------+---------------+------------------------------+ >> | | | | >> | | | | >> |14 | 15 | 17 | >> +---------------+---------------+ | >> | | | | >> | | | | >> | 4 | 12 | 6 |8 >> +---------------+---------------+------------------------------+ >> | | | | >> | | | | >> | 9 | 11 | 13 | >> +---------------+---------------+ | >> | | | | >> | | | | >> | 0 | 10 |5 | 2 >> +---------------+---------------+------------------------------+ >> >> >> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >> >> >> Number of elements in ordering 18 >> PETSc->App App->PETSc >> 0 9 0 1 >> 1 0 1 3 >> 2 10 2 4 >> 3 1 3 7 >> 4 2 4 12 >> 5 11 5 14 >> 6 12 6 15 >> 7 3 7 16 >> 8 13 8 17 >> 9 14 9 0 >> 10 15 10 2 >> 11 16 11 5 >> 12 4 12 6 >> 13 17 13 8 >> 14 5 14 9 >> 15 6 15 10 >> 16 7 16 11 >> 17 8 17 13 >> >> Now I have two questions/concerns: >> >> 1) Do processors always have the nodes in contiguous chunks of PETSc >> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >> ordering does not seem to be "good" for this grid since it seems to cross >> too many edges in the graph (here 13 edges) and by just looking at the graph >> I can(at least) think of a better distribution with only 6 edge cuts. (if >> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >> rank 1). >> > > Yes, the PETSc ordering is always contiguous. Perhaps you are not providing > the graph you think you are for partitioning. > > >> 2) Isn't it true that the final distribution should be independent of >> initial grid numbering? When I try the same grid but with the following >> (hand-generated) numbering: >> >> 14 15 16 17 >> +---------------+---------------+------------------------------+ >> | | | | >> | | | | >> |11 | 12 | 13 | >> +---------------+---------------+ | >> | | | | >> | | | | >> | 7 | 8 | 9 |10 >> +---------------+---------------+------------------------------+ >> | | | | >> | | | | >> | 4 | 5 | 6 | >> +---------------+---------------+ | >> | | | | >> | | | | >> | 0 | 1 |2 | 3 >> +---------------+---------------+------------------------------+ >> >> I get the following AO: >> >> Number of elements in ordering 18 >> PETSc->App App->PETSc >> 0 9 0 9 >> 1 10 1 10 >> 2 11 2 11 >> 3 12 3 12 >> 4 13 4 13 >> 5 14 5 14 >> 6 15 6 15 >> 7 16 7 16 >> 8 17 8 17 >> 9 0 9 0 >> 10 1 10 1 >> 11 2 11 2 >> 12 3 12 3 >> 13 4 13 4 >> 14 5 14 5 >> 15 6 15 6 >> 16 7 16 7 >> 17 8 17 8 >> >> >> which is simply the initial ordering with a change in the order in which >> processors handle nodes. Could it be that the partitioning is not unique >> and each time the algorithm only tries to obtain the "best" possible >> ordering depending on the initial distribution? If so, how should I know >> what ordering to start with? >> > > Yes, ParMetis does not provide a unique "best" ordering, which is at least > NP-complete if not worse. > > Matt > > >> I am really confused and would appreciate if someone could provide some >> insights. >> >> Thanks, >> Mohammad >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Jul 29 00:01:04 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 29 Jul 2011 05:01:04 +0000 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 4:52 AM, Mohammad Mirzadeh wrote: > Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est uses > parallel octrees/quadtrees but for what I intend to do I only need to > distribute a single tree that is created in serial among processors. > I definitely like to have the tree data-structure in parallel but that would > be another project. I also looked into Dendro and they kind of follow the > same strategy. i.e every single processor has a local copy of the whole > tree. What they do differently, however, is they somehow manage to use DA > instead of a general unstructured numbering which is quite interesting but I > still don't know how they do it. Unfortunately, they do not handle (as far > as I understood from their manual) non-graded trees which are the ones I > work with. > > So, all I need to do is to somehow distribute my grid among processors and > since each one has a local copy of data-structure I could get around the > problem. Just anotehr question. If the partitioning is not unique, do you at > least get a better numbering than the tree you start with? > You should, which is why I suggested that you are not giving the input you think you are. Matt > Mohammad > > > On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: > >> On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh wrote: >> >>> Hi all, >>> >>> I am trying to write a code to do parallel computation on quadtree >>> adaptive grids and to do so , I need to distribute the grid in parallel. I >>> have selected a general unstructured framework for telling PETSc about my >>> node numbering. An example of such grid is schematically shown below. >>> >> >> 0) If you are doing this, I think you should at least look at the p4est >> package before proceeding. >> >> >>> >>> 1 16 7 3 >>> +---------------+---------------+------------------------------+ >>> | | | | >>> | | | | >>> |14 | 15 | 17 | >>> +---------------+---------------+ | >>> | | | | >>> | | | | >>> | 4 | 12 | 6 |8 >>> +---------------+---------------+------------------------------+ >>> | | | | >>> | | | | >>> | 9 | 11 | 13 | >>> +---------------+---------------+ | >>> | | | | >>> | | | | >>> | 0 | 10 |5 | 2 >>> +---------------+---------------+------------------------------+ >>> >>> >>> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >>> >>> >>> Number of elements in ordering 18 >>> PETSc->App App->PETSc >>> 0 9 0 1 >>> 1 0 1 3 >>> 2 10 2 4 >>> 3 1 3 7 >>> 4 2 4 12 >>> 5 11 5 14 >>> 6 12 6 15 >>> 7 3 7 16 >>> 8 13 8 17 >>> 9 14 9 0 >>> 10 15 10 2 >>> 11 16 11 5 >>> 12 4 12 6 >>> 13 17 13 8 >>> 14 5 14 9 >>> 15 6 15 10 >>> 16 7 16 11 >>> 17 8 17 13 >>> >>> Now I have two questions/concerns: >>> >>> 1) Do processors always have the nodes in contiguous chunks of PETSc >>> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >>> ordering does not seem to be "good" for this grid since it seems to cross >>> too many edges in the graph (here 13 edges) and by just looking at the graph >>> I can(at least) think of a better distribution with only 6 edge cuts. (if >>> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >>> rank 1). >>> >> >> Yes, the PETSc ordering is always contiguous. Perhaps you are not >> providing the graph you think you are for partitioning. >> >> >>> 2) Isn't it true that the final distribution should be independent of >>> initial grid numbering? When I try the same grid but with the following >>> (hand-generated) numbering: >>> >>> 14 15 16 17 >>> +---------------+---------------+------------------------------+ >>> | | | | >>> | | | | >>> |11 | 12 | 13 | >>> +---------------+---------------+ | >>> | | | | >>> | | | | >>> | 7 | 8 | 9 |10 >>> +---------------+---------------+------------------------------+ >>> | | | | >>> | | | | >>> | 4 | 5 | 6 | >>> +---------------+---------------+ | >>> | | | | >>> | | | | >>> | 0 | 1 |2 | 3 >>> +---------------+---------------+------------------------------+ >>> >>> I get the following AO: >>> >>> Number of elements in ordering 18 >>> PETSc->App App->PETSc >>> 0 9 0 9 >>> 1 10 1 10 >>> 2 11 2 11 >>> 3 12 3 12 >>> 4 13 4 13 >>> 5 14 5 14 >>> 6 15 6 15 >>> 7 16 7 16 >>> 8 17 8 17 >>> 9 0 9 0 >>> 10 1 10 1 >>> 11 2 11 2 >>> 12 3 12 3 >>> 13 4 13 4 >>> 14 5 14 5 >>> 15 6 15 6 >>> 16 7 16 7 >>> 17 8 17 8 >>> >>> >>> which is simply the initial ordering with a change in the order in which >>> processors handle nodes. Could it be that the partitioning is not unique >>> and each time the algorithm only tries to obtain the "best" possible >>> ordering depending on the initial distribution? If so, how should I know >>> what ordering to start with? >>> >> >> Yes, ParMetis does not provide a unique "best" ordering, which is at least >> NP-complete if not worse. >> >> Matt >> >> >>> I am really confused and would appreciate if someone could provide some >>> insights. >>> >>> Thanks, >>> Mohammad >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Fri Jul 29 00:11:55 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Thu, 28 Jul 2011 22:11:55 -0700 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: I see. Thanks for the help Matt On Thu, Jul 28, 2011 at 10:01 PM, Matthew Knepley wrote: > On Fri, Jul 29, 2011 at 4:52 AM, Mohammad Mirzadeh wrote: > >> Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est >> uses parallel octrees/quadtrees but for what I intend to do I only need to >> distribute a single tree that is created in serial among processors. >> I definitely like to have the tree data-structure in parallel but that would >> be another project. I also looked into Dendro and they kind of follow the >> same strategy. i.e every single processor has a local copy of the whole >> tree. What they do differently, however, is they somehow manage to use DA >> instead of a general unstructured numbering which is quite interesting but I >> still don't know how they do it. Unfortunately, they do not handle (as far >> as I understood from their manual) non-graded trees which are the ones I >> work with. >> >> So, all I need to do is to somehow distribute my grid among processors and >> since each one has a local copy of data-structure I could get around the >> problem. Just anotehr question. If the partitioning is not unique, do you at >> least get a better numbering than the tree you start with? >> > > You should, which is why I suggested that you are not giving the input you > think you are. > > Matt > > >> Mohammad >> >> >> On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: >> >>> On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh wrote: >>> >>>> Hi all, >>>> >>>> I am trying to write a code to do parallel computation on quadtree >>>> adaptive grids and to do so , I need to distribute the grid in parallel. I >>>> have selected a general unstructured framework for telling PETSc about my >>>> node numbering. An example of such grid is schematically shown below. >>>> >>> >>> 0) If you are doing this, I think you should at least look at the p4est >>> package before proceeding. >>> >>> >>>> >>>> 1 16 7 3 >>>> +---------------+---------------+------------------------------+ >>>> | | | | >>>> | | | | >>>> |14 | 15 | 17 | >>>> +---------------+---------------+ | >>>> | | | | >>>> | | | | >>>> | 4 | 12 | 6 |8 >>>> +---------------+---------------+------------------------------+ >>>> | | | | >>>> | | | | >>>> | 9 | 11 | 13 | >>>> +---------------+---------------+ | >>>> | | | | >>>> | | | | >>>> | 0 | 10 |5 | 2 >>>> +---------------+---------------+------------------------------+ >>>> >>>> >>>> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >>>> >>>> >>>> Number of elements in ordering 18 >>>> PETSc->App App->PETSc >>>> 0 9 0 1 >>>> 1 0 1 3 >>>> 2 10 2 4 >>>> 3 1 3 7 >>>> 4 2 4 12 >>>> 5 11 5 14 >>>> 6 12 6 15 >>>> 7 3 7 16 >>>> 8 13 8 17 >>>> 9 14 9 0 >>>> 10 15 10 2 >>>> 11 16 11 5 >>>> 12 4 12 6 >>>> 13 17 13 8 >>>> 14 5 14 9 >>>> 15 6 15 10 >>>> 16 7 16 11 >>>> 17 8 17 13 >>>> >>>> Now I have two questions/concerns: >>>> >>>> 1) Do processors always have the nodes in contiguous chunks of PETSc >>>> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >>>> ordering does not seem to be "good" for this grid since it seems to cross >>>> too many edges in the graph (here 13 edges) and by just looking at the graph >>>> I can(at least) think of a better distribution with only 6 edge cuts. (if >>>> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >>>> rank 1). >>>> >>> >>> Yes, the PETSc ordering is always contiguous. Perhaps you are not >>> providing the graph you think you are for partitioning. >>> >>> >>>> 2) Isn't it true that the final distribution should be independent of >>>> initial grid numbering? When I try the same grid but with the following >>>> (hand-generated) numbering: >>>> >>>> 14 15 16 17 >>>> +---------------+---------------+------------------------------+ >>>> | | | | >>>> | | | | >>>> |11 | 12 | 13 | >>>> +---------------+---------------+ | >>>> | | | | >>>> | | | | >>>> | 7 | 8 | 9 |10 >>>> +---------------+---------------+------------------------------+ >>>> | | | | >>>> | | | | >>>> | 4 | 5 | 6 | >>>> +---------------+---------------+ | >>>> | | | | >>>> | | | | >>>> | 0 | 1 |2 | 3 >>>> +---------------+---------------+------------------------------+ >>>> >>>> I get the following AO: >>>> >>>> Number of elements in ordering 18 >>>> PETSc->App App->PETSc >>>> 0 9 0 9 >>>> 1 10 1 10 >>>> 2 11 2 11 >>>> 3 12 3 12 >>>> 4 13 4 13 >>>> 5 14 5 14 >>>> 6 15 6 15 >>>> 7 16 7 16 >>>> 8 17 8 17 >>>> 9 0 9 0 >>>> 10 1 10 1 >>>> 11 2 11 2 >>>> 12 3 12 3 >>>> 13 4 13 4 >>>> 14 5 14 5 >>>> 15 6 15 6 >>>> 16 7 16 7 >>>> 17 8 17 8 >>>> >>>> >>>> which is simply the initial ordering with a change in the order in which >>>> processors handle nodes. Could it be that the partitioning is not unique >>>> and each time the algorithm only tries to obtain the "best" possible >>>> ordering depending on the initial distribution? If so, how should I know >>>> what ordering to start with? >>>> >>> >>> Yes, ParMetis does not provide a unique "best" ordering, which is at >>> least NP-complete if not worse. >>> >>> Matt >>> >>> >>>> I am really confused and would appreciate if someone could provide some >>>> insights. >>>> >>>> Thanks, >>>> Mohammad >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jchludzinski at gmail.com Fri Jul 29 02:56:40 2011 From: jchludzinski at gmail.com (John Chludzinski) Date: Fri, 29 Jul 2011 03:56:40 -0400 Subject: [petsc-users] Sometimes it's better NOT to parallelize ??? (SLEPc question?) Message-ID: These are the resulting stats from decomposing a 4002x4002 (dense matrices) generalized eigenvalue problem into 2 MPI processes. Note the amount of message traffic: Max Max/Min Avg Total MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 Total # of messages: 1.602e+04 with an average message length: 2.799e+0. With the number of MPI set to 1, you get (not surprisingly): Max Max/Min Avg Total MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 In the end the time require to solve my 4002x4002 eigenvalue problem for 2 MPI processes: 1.821e+03 vs. 1.312e+03 for 1 MPI process. Am I reading this correctly? ---John *Complete stats for 2 MPI process run:* Using Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 2010 Max Max/Min Avg Total Time (sec): 1.821e+03 1.00064 1.820e+03 Objects: 2.005e+04 1.00000 2.005e+04 Flops: 1.282e+11 1.00000 1.282e+11 2.564e+11 Flops/sec: 7.046e+07 1.00064 7.044e+07 1.409e+08 Memory: 1.286e+09 1.00000 2.571e+09 MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 MPI Reductions: 2.412e+04 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.8203e+03 100.0% 2.5645e+11 100.0% 1.602e+04 100.0% 2.799e+04 100.0% 2.007e+04 83.2% *Complete stats for 1 MPI process run:* Max Max/Min Avg Total Time (sec): 1.312e+03 1.00000 1.312e+03 Objects: 2.003e+04 1.00000 2.003e+04 Flops: 2.564e+11 1.00000 2.564e+11 2.564e+11 Flops/sec: 1.955e+08 1.00000 1.955e+08 1.955e+08 Memory: 1.029e+09 1.00000 1.029e+09 MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Reductions: 2.404e+04 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.3119e+03 100.0% 2.5645e+11 100.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.002e+04 83.3% -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Fri Jul 29 03:19:31 2011 From: jroman at dsic.upv.es (Jose E. Roman) Date: Fri, 29 Jul 2011 10:19:31 +0200 Subject: [petsc-users] Sometimes it's better NOT to parallelize ??? (SLEPc question?) In-Reply-To: References: Message-ID: <780E56B7-C978-421A-BD1B-0CB1CCB161EC@dsic.upv.es> As stated in the documentation, the 'lapack' solver is a wrapper to LAPACK functions. LAPACK is a sequential library, so do not expect speedup for more than one process. The LAPACK wrappers are provided for convenience, as a debugging tool, for small problems. SLEPc is intended for large-scale sparse eigenproblems. We do not provide parallel eigensolvers for dense matrices. Jose El 29/07/2011, a las 09:56, John Chludzinski escribi?: > These are the resulting stats from decomposing a 4002x4002 (dense matrices) generalized eigenvalue problem into 2 MPI processes. Note the amount of message traffic: > > Max Max/Min Avg Total > MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 > MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 > > Total # of messages: 1.602e+04 with an average message length: 2.799e+0. > > > With the number of MPI set to 1, you get (not surprisingly): > > Max Max/Min Avg Total > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > In the end the time require to solve my 4002x4002 eigenvalue problem for 2 MPI processes: 1.821e+03 vs. 1.312e+03 for 1 MPI process. > > Am I reading this correctly? > > > ---John > > > Complete stats for 2 MPI process run: > > Using Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 2010 > > Max Max/Min Avg Total > Time (sec): 1.821e+03 1.00064 1.820e+03 > Objects: 2.005e+04 1.00000 2.005e+04 > Flops: 1.282e+11 1.00000 1.282e+11 2.564e+11 > Flops/sec: 7.046e+07 1.00064 7.044e+07 1.409e+08 > Memory: 1.286e+09 1.00000 2.571e+09 > MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 > MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 > MPI Reductions: 2.412e+04 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N --> 2N flops > and VecAXPY() for complex vectors of length N --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts %Total Avg %Total counts %Total > 0: Main Stage: 1.8203e+03 100.0% 2.5645e+11 100.0% 1.602e+04 100.0% 2.799e+04 100.0% 2.007e+04 83.2% > > > Complete stats for 1 MPI process run: > > Max Max/Min Avg Total > Time (sec): 1.312e+03 1.00000 1.312e+03 > Objects: 2.003e+04 1.00000 2.003e+04 > Flops: 2.564e+11 1.00000 2.564e+11 2.564e+11 > Flops/sec: 1.955e+08 1.00000 1.955e+08 1.955e+08 > Memory: 1.029e+09 1.00000 1.029e+09 > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Reductions: 2.404e+04 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N --> 2N flops > and VecAXPY() for complex vectors of length N --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts %Total Avg %Total counts %Total > 0: Main Stage: 1.3119e+03 100.0% 2.5645e+11 100.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.002e+04 83.3% > > From ping.rong at tuhh.de Fri Jul 29 04:37:23 2011 From: ping.rong at tuhh.de (Ping Rong) Date: Fri, 29 Jul 2011 11:37:23 +0200 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: References: <4E2F2950.2030701@tuhh.de> <4E305497.1070405@tuhh.de> Message-ID: <4E327F53.2000500@tuhh.de> Dear Hong, I have found the problem. I don't know whether this is a bug. ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), this->same_preconditioner ? SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); I used my system matrix as the precondition matrix, where matrix->mat() and precond->mat() are in deed the same matrix. I have left the KSPSetOperators(ksp) out before the KSPSolve(..) function, so KSPSetOperators() is called only once while the petsc_linear_solver object is initialized, which eliminated the petsc error ("object is in wrong state"). Also when I used petsc default ilu preconditioner, everything is fine, also the solutions are correct. However, as long as I apply the superlu as my solver package, it overwrites the system matrix. I compared the system matrix before and after solve(). With petsc ilu, it remains identical, but with superlu ilu, the system matrix is changed to the preconditioner matrix itself. Now I simply duplicate the system matrix, and use this copy as precondition matrix. Everything works perfectly. The code I used is pretty standard, just as in ex2. It took me quite a while to realize the problem, because when the system matrix is replaced by the preconditioner matrix, you still get a "correct" solution, which is however incorrectly scaled. Ping Am 28.07.2011 18:21, schrieb Hong Zhang: > Ping : > I tested similar calling procedure using > petsc-dev/src/ksp/ksp/examples/tutorials/ex5.c > successfully. > > Try following: > 1) switch to petsc-dev. See > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html on how to > get it. > 2) send us a short and stand-alone code that reproduce the error. We > can investigate it. > > Hong > >> Dear Hong, >> >> thank you very much for your time. Can you tell me when should >> KSPSetFromOptions(ksp) be called? >> well, libMesh has a class called "petsc_linear_solver", in its original code >> KSPSetFromOptions(ksp) is called while the object is created. I couldn't get >> any output, when I run the program with the option >> >> -pc_type ilu -pc_factor_mat_solver_package superlu >> -mat_superlu_ilu_droptol 0.1 -ksp_view >> >> The actual solve() function of the "petsc_linear_solver" contains the >> similar code as in the ex2.c >> ... >> ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), >> this->same_preconditioner ? >> SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); >> CHKERRABORT(libMesh::COMM_WORLD,ierr); >> >> // Set the tolerances for the iterative solver. Use the user-supplied >> // tolerance for the relative residual& leave the others at default >> values. >> ierr = KSPSetTolerances (_ksp, tol, PETSC_DEFAULT, PETSC_DEFAULT, max_its); >> CHKERRABORT(libMesh::COMM_WORLD,ierr); >> >> ierr = KSPSetFromOptions (_ksp);<-------------- I added these two lines, >> CHKERRABORT(libMesh::COMM_WORLD,ierr);<-------------- then I got proper >> output. >> >> // Solve the linear system >> ierr = KSPSolve (_ksp, rhs->vec(), solution->vec()); >> CHKERRABORT(libMesh::COMM_WORLD,ierr); >> .... >> >> The problem is that I have to solve a multiple frequency system, and for >> each frequency I have several RHS. The same ksp context is used to solve the >> systems repeatedly. I always call KSPSetOperators() with the option >> DIFFERENT_NONZERO_PATTERN for the first RHS and SAME_PRECONDITIONER for the >> rest, since the system matrix remains the same. For the first frequency, >> everything is fine, both DIFFERENT_NONZERO_PATTERN and SAME_PRECONDITIONER >> options. I got following output from the -ksp_view. >> ... >> PC Object: >> type: ilu >> ILU: out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 1e-12 >> using diagonal shift to prevent zero pivot >> matrix ordering: natural >> factor fill ratio given 0, needed 0 >> Factored matrix follows: >> Matrix Object: >> type=seqaij, rows=2883, cols=2883 >> package used to perform factorization: superlu<---------------- >> superlu is properly set >> .... >> >> but when the second frequency comes up, KSPSetOperators() is called again >> with DIFFERENT_NONZERO_PATTERN option for the first RHS. the command line >> option doesn't seem to work anymore. >> ... >> PC Object: >> type: ilu >> ILU: out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 1e-12 >> using diagonal shift to prevent zero pivot >> matrix ordering: natural >> factor fill ratio given 1, needed 1 >> Factored matrix follows: >> Matrix Object: >> type=seqaij, rows=2883, cols=2883 >> package used to perform factorization: petsc<------------ not >> superlu anymore >> ... >> >> and then for the second RHS, when it calls KSPSetOperators(ksp) in the >> solve() routine, petsc throws the following error message. >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Object is in wrong state! >> [0]PETSC ERROR: Cannot change solver matrix package after PC has been setup >> or used! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 >> CDT 2011 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./PLTMainTest.opt on a linux-gnu named abe-vm by mubpr Wed >> Jul 27 19:41:22 2011 >> [0]PETSC ERROR: Libraries linked from >> /home/mubpr/libs/petsc/linux-gnu-acml-shared-opt/lib >> [0]PETSC ERROR: Configure run at Tue Jul 26 22:09:39 2011 >> [0]PETSC ERROR: Configure options >> --with-blas-lapack-lib=/home/mubpr/libs/acml/gfortran64_mp/lib/libacml_mp.so >> --with-blas-lapack-include=/home/mubpr/libs/acml/gfortran64_mp/include >> --with-scalar-type=complex --with-clanguage=c++ >> --with-mpi-dir=/home/mubpr/libs/mpich2 --download-superlu=yes >> --with-superlu=1 --download-parmetis=yes --with-parmetis=1 >> --download-superlu_dist=yes --with-superlu_dist=1 --download-mumps=yes >> --with-mumps=1 --download-blacs=yes --with-blacs=1 --download-scalapack=yes >> --with-scalapack=1 --download-spooles=yes --with-spooles=1 >> --download-umfpack=yes --with-umfpack=1 --with-debugging=no --with-shared=1 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: PCFactorSetMatSolverPackage_Factor() line 183 in >> src/ksp/pc/impls/factor/factimpl.c >> [0]PETSC ERROR: PCFactorSetMatSolverPackage() line 276 in >> src/ksp/pc/impls/factor/factor.c >> [0]PETSC ERROR: PCSetFromOptions_Factor() line 275 in >> src/ksp/pc/impls/factor/factimpl.c >> [0]PETSC ERROR: PCSetFromOptions_ILU() line 112 in >> src/ksp/pc/impls/factor/ilu/ilu.c >> [0]PETSC ERROR: PCSetFromOptions() line 185 in src/ksp/pc/interface/pcset.c >> [0]PETSC ERROR: KSPSetFromOptions() line 323 in src/ksp/ksp/interface/itcl.c >> [0]PETSC ERROR: User provided function() line 696 in >> "unknowndirectory/"src/solvers/petsc_linear_solver.C >> >> Any idea what the problem may be? I would be very grateful, if you can give >> me any hint. thanks alot >> >> best regards >> Ping >> >> >> >> Am 27.07.2011 04:13, schrieb Hong Zhang: >>> Did you call KSPSetFromOptions(ksp)? >>> Run your code with '-options_table' to dump list of options inputted >>> or '-options_left' to dump list of unused options. >>> >>> I tested with petsc-3.1/src/ksp/ksp/examples/tutorials/ex2.c: >>> ./ex2 >>> ... >>> SuperLU run parameters: >>> ... >>> ILU_DropTol: 0.1 >>> ILU_FillTol: 0.01 >>> ILU_FillFactor: 10 >>> ILU_DropRule: 9 >>> ILU_Norm: 2 >>> ILU_MILU: 2 >>> >>> Hong >>> >>> On Tue, Jul 26, 2011 at 3:53 PM, Ping Rong wrote: >>>> Dear developers, >>>> >>>> I have compiled petsc-3.1-p8 for a while. Now I would like to use superlu >>>> as >>>> an ilu preconditioner, since it offers the drop tolerance option. I have >>>> read in a thread >>>> >>>> (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) >>>> that one can run the code with option >>>> >>>> -pc_type ilu -pc_factor_mat_solver_package superlu >>>> -mat_superlu_ilu_droptol >>>> <> >>>> >>>> to setup the ilu preconditioner. I also use the option "-help |grep >>>> superlu" >>>> to check the settings. However, no matter how I change the value of >>>> -mat_superlu_ilu_droptol, I always got the same result >>>> ... >>>> -mat_superlu_lwork<0>: size of work array in bytes used by factorization >>>> (None) >>>> -mat_superlu_ilu_droptol<0.0001>: ILU_DropTol (None) >>>> -mat_superlu_ilu_filltol<0.01>: ILU_FillTol (None) >>>> ... >>>> I dont know whether I understand it correctly. But it seems to me the >>>> value >>>> of the droptol has never been changed. In that maillist thread, it was >>>> also >>>> mentioned that the dev-version is recommended, because of some bugs in >>>> the >>>> superlu interface. I have then compiled the current dev-version. but the >>>> problem is my program is based on another library (libMesh) which uses >>>> petsc >>>> as a solver package. Since some of the interfaces have been changed in >>>> the >>>> dev-version, I would not be able to compile the libMesh with petsc >>>> anymore. >>>> Is there anyway I can correct the superlu interface in the 3.1-p8 version >>>> directly? Any help will be appreciated!! >>>> >>>> Best regards >>>> >>>> -- >>>> Ping Rong, M.Sc. >>>> Hamburg University of Technology >>>> Institut of modelling and computation >>>> Denickestra?e 17 (Room 3031) >>>> 21073 Hamburg >>>> >>>> Tel.: ++49 - (0)40 42878 2749 >>>> Fax: ++49 - (0)40 42878 43533 >>>> Email: ping.rong at tuhh.de >>>> >>>> >> >> -- >> Ping Rong, M.Sc. >> Hamburg University of Technology >> Institut of modelling and computation >> Denickestra?e 17 (Room 3031) >> 21073 Hamburg >> >> Tel.: ++49 - (0)40 42878 2749 >> Fax: ++49 - (0)40 42878 43533 >> Email: ping.rong at tuhh.de >> >> -- Ping Rong, M.Sc. Hamburg University of Technology Institut of modelling and computation Denickestra?e 17 (Room 3031) 21073 Hamburg Tel.: ++49 - (0)40 42878 2749 Fax: ++49 - (0)40 42878 43533 Email: ping.rong at tuhh.de From Debao.Shao at brion.com Fri Jul 29 05:43:32 2011 From: Debao.Shao at brion.com (Debao Shao) Date: Fri, 29 Jul 2011 03:43:32 -0700 Subject: [petsc-users] (no subject) In-Reply-To: <715C3B78-1781-4DE4-94F0-8289C4068138@mcs.anl.gov> References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> <715C3B78-1781-4DE4-94F0-8289C4068138@mcs.anl.gov> Message-ID: <384FF55F15E3E447802DC8CCA85696980E13EB8D93@EX03> Hi, Barry: Thanks a lot for your help. I tried ILU(1) as the precondition of GMRES, it can solve the ill-conditioned problem. I added "-ksp_monitor_true_residual & -ksp_converged_reason" to monitor the problem, in general, I find ILU(1) converged within less iterations than ILU(0). Thanks, Debao -----Original Message----- From: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Barry Smith Sent: Friday, July 29, 2011 12:11 PM To: PETSc users list Subject: Re: [petsc-users] (no subject) On Jul 28, 2011, at 10:57 PM, Debao Shao wrote: > Dear all: > > I'm using PETSC iterative solver(PCILU & KSPGMRES) in OOQP, but when the to be solved matrix is ill-posed, say condition number ~ 1e+4, then, KSPGMRES will exceed its maximal iteration number(default as 10000), while when I checked the same data with cholmod direct solver, it can solve the problem and return an answer correctly. > > Does it mean the iterative solver(like GMRES) relies more on the well-pose matrix than cholmod direct solver? Yes, generally iterative solvers take a longer time for more ill-conditioned matrices. Direct solvers are not usually effected much by ill conditioning until it becomes large. > How can I improve the case to avoid KSPGMRES fail case? There are many many choices in PETSc making it hard to know how to get started on getting an efficient solver. 1) When the matrix is very ill-conditioned or small (say less than 1000 by 1000) then the best bet is a direct solver -pc_type lu (and in parallel you need to install an external solver package MUMPS or SuperLU_dist see http://www.mcs.anl.gov/petsc/petsc-as/documentation/linearsolvertable.html) 2) in your case condition number of ~ 10,000 is low enough that iterative solver can likely be faster than direct solver so it is worth investigating iterative solvers. The first thing to ask is there any special structure in the matrix you can take advantage of? Is is symmetric, positive definite then use -ksp_type cg Does the matrix arise from an elliptic PDE discretization on a mesh then likely you can get multigrid to work well either geometric or algebraic see http://www.mcs.anl.gov/petsc/petsc-as/documentation/linearsolvertable.html) . Does the matrix come from a fluids problem? Then there are various tricks we can help you with. If you know of no particular structure then the fall back is ILU. Since ILU(0) doesn't seem to work you can try ILU(k) for small k integer to see if that helps, just add the option -pc_factor_levels 1 or 2 or 3 and see how that affects the convergence. Always run with -ksp_monitor_true_residual -ksp_converged_reason when testing to see how the solver is working. Since the default GMRES restart is 30 that may be hurting your convergence, you can try a larger restart with -ksp_gmres_restart 100 or use -ksp_type bcgs that does not require a restart. This is enough to get you started. Are you interested in the parallel case? That introduces many more difficulties. Barry > > Your answer is appreciated. > > Thanks, > Debao > > -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. From knepley at gmail.com Fri Jul 29 07:35:15 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 29 Jul 2011 12:35:15 +0000 Subject: [petsc-users] Sometimes it's better NOT to parallelize ??? (SLEPc question?) In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 7:56 AM, John Chludzinski wrote: > These are the resulting stats from decomposing a 4002x4002 (dense matrices) > generalized eigenvalue problem into 2 MPI processes. Note the amount of > message traffic: The way to solve this problem is to use Elemental ( http://code.google.com/p/elemental/), which solves the dense parallel eigenproblem. Matt > > Max Max/Min Avg Total > MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 > MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 > > Total # of messages: 1.602e+04 with an average message length: 2.799e+0. > > > With the number of MPI set to 1, you get (not surprisingly): > > Max Max/Min Avg Total > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > In the end the time require to solve my 4002x4002 eigenvalue problem for 2 > MPI processes: 1.821e+03 vs. 1.312e+03 for 1 MPI process. > > Am I reading this correctly? > > > ---John > > > *Complete stats for 2 MPI process run:* > > Using Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 2010 > > Max Max/Min Avg Total > Time (sec): 1.821e+03 1.00064 1.820e+03 > Objects: 2.005e+04 1.00000 2.005e+04 > Flops: 1.282e+11 1.00000 1.282e+11 2.564e+11 > Flops/sec: 7.046e+07 1.00064 7.044e+07 1.409e+08 > Memory: 1.286e+09 1.00000 2.571e+09 > MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04 > MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08 > MPI Reductions: 2.412e+04 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N > --> 2N flops > and VecAXPY() for complex vectors of length N > --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- > -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts > %Total Avg %Total counts %Total > 0: Main Stage: 1.8203e+03 100.0% 2.5645e+11 100.0% 1.602e+04 100.0% > 2.799e+04 100.0% 2.007e+04 83.2% > > > *Complete stats for 1 MPI process run:* > > Max Max/Min Avg Total > Time (sec): 1.312e+03 1.00000 1.312e+03 > Objects: 2.003e+04 1.00000 2.003e+04 > Flops: 2.564e+11 1.00000 2.564e+11 2.564e+11 > Flops/sec: 1.955e+08 1.00000 1.955e+08 1.955e+08 > Memory: 1.029e+09 1.00000 1.029e+09 > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Reductions: 2.404e+04 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N > --> 2N flops > and VecAXPY() for complex vectors of length N > --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- > -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts > %Total Avg %Total counts %Total > 0: Main Stage: 1.3119e+03 100.0% 2.5645e+11 100.0% 0.000e+00 0.0% > 0.000e+00 0.0% 2.002e+04 83.3% > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Fri Jul 29 09:29:39 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 29 Jul 2011 09:29:39 -0500 Subject: [petsc-users] the convergence of petsc iterative solver In-Reply-To: <384FF55F15E3E447802DC8CCA85696980E13EB8C94@EX03> References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> <384FF55F15E3E447802DC8CCA85696980E13EB8C91@EX03> <384FF55F15E3E447802DC8CCA85696980E13EB8C94@EX03> Message-ID: Debao : > > I'm using PETSC iterative solver(PCILU & KSPGMRES) in OOQP, but when the to be solved matrix is ill-posed, say condition number ~ 1e+4, then, KSPGMRES will exceed its maximal iteration number(default as 10000), while when I checked the same data with cholmod direct solver, it can solve the problem and return an answer correctly. > > Does it mean the iterative solver(like GMRES) relies more on the well-pose matrix than cholmod direct solver? Yes. Iterative solvers takes advantage of sparse matrix storage and are able to solve very large systems. However, its efficiency depends on the well-pose/conditioning of matrices. Direct solver is robust, but requires matrix factorization - require large memory storage and operations. It is used for the cases that iterative solver fails. > How can I improve the case to avoid KSPGMRES fail case? Yes, you need use preconditioner. Run your code with '-help' to check available pc. Experimenting various preconditoners. Your condition number 1.e+4 is not bad. Preconditioning should work. Hong > > Your answer is appreciated. > > Thanks, > Debao > > -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. > From travis.fisher at nasa.gov Fri Jul 29 09:41:25 2011 From: travis.fisher at nasa.gov (Travis C. Fisher) Date: Fri, 29 Jul 2011 10:41:25 -0400 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems Message-ID: <4E32C695.1060806@nasa.gov> I ran into the following problem when trying to use superlu_dist as a preconditioner for matrix free SNES with GMRES in my high order Navier Stokes solver: I set the nonzero pattern of my preconditioner matrix at the begging of a simulation (t=0). The overall stencil of the numerical method never changes, so the nonzero pattern should never change. I use an implicit Runge Kutta scheme to advance the solution in time. The preconditioning matrix is the frozen jacobian at the start of a time step. The same preconditioner is used for every stage of the RK scheme. After one time step is complete, the preconditioner matrix is calculated for the new time step. The problem I am having with superlu_dist is that if I use SAME_NONZERO_PATTERN, I get a segmentation violation. However, if I use DIFFERENT_NONZERO_PATTERN, everything works fine. If I am conservative about how I initialize the nonzero pattern and desire it to never to change, why do I need to use DIFFERENT_NONZERO_PATTERN? Individual components in the nonzero structure will be zero for some steps and not others, but the overall nonzero structure is fixed. I don't get errors when using Additive Schwarz with ILU, but I am wondering if convergence problems occur because of how the nonzero patterns are handled. Any ideas of what I am doing incorrectly? What is SAME_NONZERO_PATTERN really telling the nonlinear solver over multiple calls? Most of the examples only have one call to SNESSolve, so I feel that this is not well covered elsewhere. Thanks in advance. Travis Fisher From hzhang at mcs.anl.gov Fri Jul 29 09:42:39 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 29 Jul 2011 09:42:39 -0500 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: <4E327F53.2000500@tuhh.de> References: <4E2F2950.2030701@tuhh.de> <4E305497.1070405@tuhh.de> <4E327F53.2000500@tuhh.de> Message-ID: Ping : Please send me your code. I do not understand why the system matrix is overwritten by factored matrix when superlu is used. We do not support in-place factorization for external packages. Something is incorrect here. Hong > > I have found the problem. I don't know whether this is a bug. > > ? ? ? ?ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? this->same_preconditioner ? > SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); > > > I used my system matrix as the precondition matrix, where matrix->mat() and > precond->mat() are in deed the same matrix. I have left the > KSPSetOperators(ksp) out before the KSPSolve(..) function, so > KSPSetOperators() is called only once while the petsc_linear_solver object > is initialized, which eliminated the petsc error ("object is in wrong > state"). > Also when I used petsc default ilu preconditioner, everything is fine, also > the solutions are correct. However, as long as I apply the superlu as my > solver package, it overwrites the system matrix. I compared the system > matrix before and after solve(). With petsc ilu, it remains identical, but > with superlu ilu, the system matrix is changed to the preconditioner matrix > itself. Now I simply duplicate the system matrix, and use this copy as > precondition matrix. Everything works perfectly. > > The code I used is pretty standard, just as in ex2. It took me quite a while > to realize the problem, because when the system matrix is replaced by the > preconditioner matrix, you still get a "correct" solution, which is however > incorrectly scaled. > > Ping > > > Am 28.07.2011 18:21, schrieb Hong Zhang: >> >> Ping : >> I tested similar calling procedure using >> petsc-dev/src/ksp/ksp/examples/tutorials/ex5.c >> successfully. >> >> Try following: >> 1) switch to petsc-dev. See >> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html on how to >> get it. >> 2) send us a short and stand-alone code that reproduce the error. We >> can investigate it. >> >> Hong >> >>> Dear Hong, >>> >>> thank you very much for your time. Can you tell me when should >>> ?KSPSetFromOptions(ksp) be called? >>> well, libMesh has a class called "petsc_linear_solver", in its original >>> code >>> KSPSetFromOptions(ksp) is called while the object is created. I couldn't >>> get >>> any output, when I run the program with the option >>> >>> ? -pc_type ilu -pc_factor_mat_solver_package superlu >>> -mat_superlu_ilu_droptol 0.1 -ksp_view >>> >>> The actual solve() function of the "petsc_linear_solver" contains the >>> similar code as in the ex2.c >>> ... >>> ? ?ierr = KSPSetOperators(_ksp, matrix->mat(), precond->mat(), >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? this->same_preconditioner ? >>> SAME_PRECONDITIONER : DIFFERENT_NONZERO_PATTERN); >>> ? ?CHKERRABORT(libMesh::COMM_WORLD,ierr); >>> >>> ?// Set the tolerances for the iterative solver. ?Use the user-supplied >>> ?// tolerance for the relative residual& ?leave the others at default >>> values. >>> ?ierr = KSPSetTolerances (_ksp, tol, PETSC_DEFAULT, PETSC_DEFAULT, >>> max_its); >>> ?CHKERRABORT(libMesh::COMM_WORLD,ierr); >>> >>> ?ierr = KSPSetFromOptions (_ksp);<-------------- ?I added these two >>> lines, >>> ?CHKERRABORT(libMesh::COMM_WORLD,ierr);<-------------- ?then I got proper >>> output. >>> >>> ?// Solve the linear system >>> ?ierr = KSPSolve (_ksp, rhs->vec(), solution->vec()); >>> ?CHKERRABORT(libMesh::COMM_WORLD,ierr); >>> .... >>> >>> The problem is that I have to solve a multiple frequency system, and for >>> each frequency I have several RHS. The same ksp context is used to solve >>> the >>> systems repeatedly. I always call KSPSetOperators() with the option >>> DIFFERENT_NONZERO_PATTERN for the first RHS and SAME_PRECONDITIONER for >>> the >>> rest, since the system matrix remains the same. ?For the first frequency, >>> everything is fine, both DIFFERENT_NONZERO_PATTERN and >>> SAME_PRECONDITIONER >>> options. I got following output from the -ksp_view. >>> ... >>> PC Object: >>> ?type: ilu >>> ? ?ILU: out-of-place factorization >>> ? ?0 levels of fill >>> ? ?tolerance for zero pivot 1e-12 >>> ? ?using diagonal shift to prevent zero pivot >>> ? ?matrix ordering: natural >>> ? ?factor fill ratio given 0, needed 0 >>> ? ? ?Factored matrix follows: >>> ? ? ? ?Matrix Object: >>> ? ? ? ? ?type=seqaij, rows=2883, cols=2883 >>> ? ? ? ? ?package used to perform factorization: superlu<---------------- >>> superlu is properly set >>> .... >>> >>> but when the second frequency comes up, KSPSetOperators() is called again >>> with DIFFERENT_NONZERO_PATTERN option for the first RHS. the command line >>> option doesn't seem to work anymore. >>> ... >>> PC Object: >>> ?type: ilu >>> ? ?ILU: out-of-place factorization >>> ? ?0 levels of fill >>> ? ?tolerance for zero pivot 1e-12 >>> ? ?using diagonal shift to prevent zero pivot >>> ? ?matrix ordering: natural >>> ? ?factor fill ratio given 1, needed 1 >>> ? ? ?Factored matrix follows: >>> ? ? ? ?Matrix Object: >>> ? ? ? ? ?type=seqaij, rows=2883, cols=2883 >>> ? ? ? ? ?package used to perform factorization: petsc<------------ not >>> superlu anymore >>> ... >>> >>> and then for the second RHS, when it calls KSPSetOperators(ksp) in the >>> solve() routine, petsc throws the following error message. >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: Object is in wrong state! >>> [0]PETSC ERROR: Cannot change solver matrix package after PC has been >>> setup >>> or used! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 >>> CDT 2011 >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [0]PETSC ERROR: See docs/index.html for manual pages. >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: ./PLTMainTest.opt on a linux-gnu named abe-vm by mubpr >>> Wed >>> Jul 27 19:41:22 2011 >>> [0]PETSC ERROR: Libraries linked from >>> /home/mubpr/libs/petsc/linux-gnu-acml-shared-opt/lib >>> [0]PETSC ERROR: Configure run at Tue Jul 26 22:09:39 2011 >>> [0]PETSC ERROR: Configure options >>> >>> --with-blas-lapack-lib=/home/mubpr/libs/acml/gfortran64_mp/lib/libacml_mp.so >>> --with-blas-lapack-include=/home/mubpr/libs/acml/gfortran64_mp/include >>> --with-scalar-type=complex --with-clanguage=c++ >>> --with-mpi-dir=/home/mubpr/libs/mpich2 --download-superlu=yes >>> --with-superlu=1 --download-parmetis=yes --with-parmetis=1 >>> --download-superlu_dist=yes --with-superlu_dist=1 --download-mumps=yes >>> --with-mumps=1 --download-blacs=yes --with-blacs=1 >>> --download-scalapack=yes >>> --with-scalapack=1 --download-spooles=yes --with-spooles=1 >>> --download-umfpack=yes --with-umfpack=1 --with-debugging=no >>> --with-shared=1 >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: PCFactorSetMatSolverPackage_Factor() line 183 in >>> src/ksp/pc/impls/factor/factimpl.c >>> [0]PETSC ERROR: PCFactorSetMatSolverPackage() line 276 in >>> src/ksp/pc/impls/factor/factor.c >>> [0]PETSC ERROR: PCSetFromOptions_Factor() line 275 in >>> src/ksp/pc/impls/factor/factimpl.c >>> [0]PETSC ERROR: PCSetFromOptions_ILU() line 112 in >>> src/ksp/pc/impls/factor/ilu/ilu.c >>> [0]PETSC ERROR: PCSetFromOptions() line 185 in >>> src/ksp/pc/interface/pcset.c >>> [0]PETSC ERROR: KSPSetFromOptions() line 323 in >>> src/ksp/ksp/interface/itcl.c >>> [0]PETSC ERROR: User provided function() line 696 in >>> "unknowndirectory/"src/solvers/petsc_linear_solver.C >>> >>> Any idea what the problem may be? I would be very grateful, if you can >>> give >>> me any hint. thanks alot >>> >>> best regards >>> Ping >>> >>> >>> >>> Am 27.07.2011 04:13, schrieb Hong Zhang: >>>> >>>> Did you call KSPSetFromOptions(ksp)? >>>> Run your code with '-options_table' to dump list of options inputted >>>> or '-options_left' to dump list of unused options. >>>> >>>> I tested with petsc-3.1/src/ksp/ksp/examples/tutorials/ex2.c: >>>> ./ex2 >>>> ... >>>> ? ? ? ? ? ? SuperLU run parameters: >>>> ? ? ? ? ? ? ... >>>> ? ? ? ? ? ? ? ILU_DropTol: 0.1 >>>> ? ? ? ? ? ? ? ILU_FillTol: 0.01 >>>> ? ? ? ? ? ? ? ILU_FillFactor: 10 >>>> ? ? ? ? ? ? ? ILU_DropRule: 9 >>>> ? ? ? ? ? ? ? ILU_Norm: 2 >>>> ? ? ? ? ? ? ? ILU_MILU: 2 >>>> >>>> Hong >>>> >>>> On Tue, Jul 26, 2011 at 3:53 PM, Ping Rong ? ?wrote: >>>>> >>>>> Dear developers, >>>>> >>>>> I have compiled petsc-3.1-p8 for a while. Now I would like to use >>>>> superlu >>>>> as >>>>> an ilu preconditioner, since it offers the drop tolerance option. I >>>>> have >>>>> read in a thread >>>>> >>>>> >>>>> (https://lists.mcs.anl.gov/mailman/htdig/petsc-users/2010-December/007439.html) >>>>> that one can run the code with option >>>>> >>>>> -pc_type ilu -pc_factor_mat_solver_package superlu >>>>> -mat_superlu_ilu_droptol >>>>> <> >>>>> >>>>> to setup the ilu preconditioner. I also use the option "-help |grep >>>>> superlu" >>>>> to check the settings. However, no matter how I change the value of >>>>> -mat_superlu_ilu_droptol, I always got the same result >>>>> ... >>>>> ?-mat_superlu_lwork<0>: size of work array in bytes used by >>>>> factorization >>>>> (None) >>>>> ?-mat_superlu_ilu_droptol<0.0001>: ILU_DropTol (None) >>>>> ?-mat_superlu_ilu_filltol<0.01>: ILU_FillTol (None) >>>>> ... >>>>> I dont know whether I understand it correctly. But it seems to me the >>>>> value >>>>> of the droptol has never been changed. In that maillist thread, it was >>>>> also >>>>> mentioned that the dev-version is recommended, because of some bugs in >>>>> the >>>>> superlu interface. I have then compiled the current dev-version. but >>>>> the >>>>> problem is my program is based on another library (libMesh) which uses >>>>> petsc >>>>> as a solver package. Since some of the interfaces have been changed in >>>>> the >>>>> dev-version, I would not be able to compile the libMesh with petsc >>>>> anymore. >>>>> Is there anyway I can correct the superlu interface in the 3.1-p8 >>>>> version >>>>> directly? Any help will be appreciated!! >>>>> >>>>> Best regards >>>>> >>>>> -- >>>>> Ping Rong, M.Sc. >>>>> Hamburg University of Technology >>>>> Institut of modelling and computation >>>>> Denickestra?e 17 (Room 3031) >>>>> 21073 Hamburg >>>>> >>>>> Tel.: ++49 - (0)40 42878 2749 >>>>> Fax: ?++49 - (0)40 42878 43533 >>>>> Email: ping.rong at tuhh.de >>>>> >>>>> >>> >>> -- >>> Ping Rong, M.Sc. >>> Hamburg University of Technology >>> Institut of modelling and computation >>> Denickestra?e 17 (Room 3031) >>> 21073 Hamburg >>> >>> Tel.: ++49 - (0)40 42878 2749 >>> Fax: ?++49 - (0)40 42878 43533 >>> Email: ping.rong at tuhh.de >>> >>> > > > -- > Ping Rong, M.Sc. > Hamburg University of Technology > Institut of modelling and computation > Denickestra?e 17 (Room 3031) > 21073 Hamburg > > Tel.: ++49 - (0)40 42878 2749 > Fax: ?++49 - (0)40 42878 43533 > Email: ping.rong at tuhh.de > > From hzhang at mcs.anl.gov Fri Jul 29 09:59:00 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 29 Jul 2011 09:59:00 -0500 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: <4E32C695.1060806@nasa.gov> References: <4E32C695.1060806@nasa.gov> Message-ID: Travis : "SAME_NONZERO_PATTERN" should be used in your case. It is likely a bug that causes the segmentation violation. Which packages do you use, superlu (sequential) or superlu_dist (parallel)? superlu_dist does not support ilu. Are you using latest petsc-dev? We fixed bugs in petsc-dev and reset default superlu_dist options. Please try 1) use petsc-dev 2) turn off superlu_dist options with runtime options '-mat_superlu_dist_equil NO -mat_superlu_dist_rowperm NATURAL -mat_superlu_dist_colperm NATURAL ' 3) try parallel direct solver mumps Let us know what you get. Hong > I ran into the following problem when trying to use superlu_dist as a > preconditioner for matrix free SNES with GMRES in my high order Navier > Stokes solver: > > I set the nonzero pattern of my preconditioner matrix at the begging of a > simulation (t=0). The overall stencil of the numerical method never changes, > so the nonzero pattern should never change. I use an implicit Runge Kutta > scheme to advance the solution in time. The preconditioning matrix is the > frozen jacobian at the start of a time step. The same preconditioner is used > for every stage of the RK scheme. After one time step is complete, the > preconditioner matrix is calculated for the new time step. > > The problem I am having with superlu_dist is that if I use > SAME_NONZERO_PATTERN, I get a segmentation violation. However, if I use > DIFFERENT_NONZERO_PATTERN, everything works fine. If I am conservative about > how I initialize the nonzero pattern and desire it to never to change, why > do I need to use DIFFERENT_NONZERO_PATTERN? Individual components in the > nonzero structure will be zero for some steps and not others, but the > overall nonzero structure is fixed. I don't get errors when using Additive > Schwarz with ILU, but I am wondering if convergence problems occur because > of how the nonzero patterns are handled. Any ideas of what I am doing > incorrectly? What is SAME_NONZERO_PATTERN really telling the nonlinear > solver over multiple calls? Most of the examples only have one call to > SNESSolve, so I feel that this is not well covered elsewhere. > > Thanks in advance. > > Travis Fisher > > From sperif at gmail.com Fri Jul 29 10:10:12 2011 From: sperif at gmail.com (Pierre-Yves Aquilanti) Date: Fri, 29 Jul 2011 17:10:12 +0200 Subject: [petsc-users] force ksp to continue if divergence or increase divergence tol, how ? Message-ID: Hello, I'm using a preconditionner of my construction which has the particularity to make the norm of the residual vector increase suddently (from 1e-1 to 1e+6). The problem is that if that PETSc concludes to divergence after applying my preconditionner. Also, i setted the -ksp_divtol flag to 100 (random value to make it works) and also used the KSPSetTolerances function with the same value. I'm probably missing some point but can't figure out which one. Do someone has an idea ? Thanks a lot. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Fri Jul 29 10:35:07 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 29 Jul 2011 10:35:07 -0500 Subject: [petsc-users] memory corruption In-Reply-To: References: <31AEC7FB-52E7-4437-A16E-5452DE6419DB@lbl.gov> Message-ID: Rebecca: > Yes, there is no memory corruption for small size problem test after I add those options. I have not tested on big size problems yet. Great! I suggest you also install mumps. Although this might be inappropriate for your current position :-( you have an additional solver for debugging. When superlu crashes, switch to mumps for investigation. > > For using superlu_dist as a direct solver, I would like to make sure that the there is no memory corruption for large size problem, i.e., the memory requirement for a single processor does not exceed the memory of each processor. You must experiment. The memory analysis below is based on the size of the original matrix, not the factored matrix, which could be many times larger than the original one. Option '-mat_superlu_dist_statprint' gives memory usage in superlu_dist. Hong > > For example, on Hopper, 1 node has 24 cores, and if I have 24 cores per node, each core has the memory approximately 1.33 Gb. > > Assume the problem size is M*N, and there are 4 unknowns per grid point, and the standard 13-point scheme is used for discretization. Therefore, the number of non zeroes per row is 13. > > The sparse matrix is 4MN X 4MN in size with 13*4MN non zeroes. > > Assume that the data type is PetscScalar (8 bytes), what would be the memory usage for the matrix? > > I ran a test problem of 64X64 for 10 time steps in np=1,2,4,8,16,32. > > With the help of '-log_summary', I was able to tell the memory usage, for example, when np=32, > > > ? ? ? ? ? ? ?Matrix ? ? 9 ? ? ? ? ? ? ?9 ? ? ?1172968 ? ? 0 > > > The memory usage for processor 0 is 1,172,968 bytes in Matrix. > > Is this 1,172,968 bytes the total number for those 9 creations/destructions, or how could I understand this 1,172,968 come from? > > I did the math as follow, the non zeroes of the matrix is 13*4MN=212,992; > > and the memory is 8(bytes for PetscScalar) * 212,992 = 1,703,936; > > on each processor, the memory is 1,703,936/32 = 53,248 > > this 53,248 is far from 1,172,968. > > even if I multiply by 10 (for 10 time steps iterations), 530,248 is still far from 1,172,958. > > Where did I get this number wrong? > > ?-log_summary returns: > > np=1 ? ? ? ? ? ? ?Matrix ? ? 3 ? ? ? ? ? ? ?3 ? ? 38785428 ? ? 0 > np=2 ? ? ? ? ? ?Matrix ? ? 9 ? ? ? ? ? ? ?9 ? ? 19871976 ? ? 0 > np=4 ? ? ? ? ? ?Matrix ? ? 9 ? ? ? ? ? ? ?9 ? ? ?9948136 ? ? 0 > np=8 ? ? ? ? ? ?Matrix ? ?9 ? ? ? ? ? ? ? 9 ? ? ? 4894696 0 > np=16 ? ? ? ? ? ? ?Matrix ? ? 9 ? ? ? ? ? ? ?9 ? ? ?2413544 ? ? 0 > np=32 ? ? ? ? ? ?Matrix ? ? 9 ? ? ? ? ? ? ?9 ? ? ?1172968 ? ? 0 > > for other cases. > > Thanks very much! > > Best regards, > > Rebecca > > > On Jul 28, 2011, at 1:05 PM, Hong Zhang wrote: > >> Rebecca: >> >> Turn off orderings and some options, e.g., >> -mat_superlu_dist_equil NO -mat_superlu_dist_rowperm NATURAL >> -mat_superlu_dist_colperm NATURAL >> >> Do you still get memory corruption? >> >> Hong >> >>> Hello all, >>> >>> I tried to use superlu as a direct solver running on Hopper, but found that there are some memory corruption errors: >>> >>> x/xyuan> cd $PBS_O_WORKDIR >>> Directory: /global/homes/x/xyuan/Workspace_Nersc/cartmhdpdslin/trunk/test_superlu_as_direct_solver/m256_p1024 >>> test_superlu_as_direct_solver/m256_p1024> aprun -n 1024 ./twcartffxmhd.exe -options_file option_twcartffxmhd_256 >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25137 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25146 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25144 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25133 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25136 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25142 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25145 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25148 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25149 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25147 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25135 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25134 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25138 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25141 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25140 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25139 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25132 : Permission denied >>> [0] ERROR - MPIU_nem_gni_get_hugepages(): Can't create file /var/lib/hugetlbfs/global/pagesize-2097152/hugepagefile.MPICH.1.25143 : Permission denied >>> ********************************************* >>> cartesian coordinate code(np = 1024) >>> start time = 0.0000 >>> time accuracy order = 2 >>> viscosity = 0.0500 >>> resistivity = 0.0050 >>> skin depth = 1.0000 >>> hyper resistivity = 0.00000630 >>> hyper viscosity = 0.00503929 >>> problem size: 256 by 256 >>> dt = 0.1000 >>> ********************************************* >>> ******* start solving for time = 0.10000 at time step = 1****** >>> ?0 SNES Function norm 6.220836330249e-03 >>> Linear solve converged due to CONVERGED_ITS iterations 1 >>> ?1 SNES Function norm 3.041982522542e-07 >>> *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe*** glibc detected *** *** glibc detected *** *** glibc detected *** ./twcartffxmhd.exe: malloc(): memory corruption: 0x0000000001e31280 *** >>> >>> Any idea what is wrong here? >>> >>> Thanks very much! >>> >>> Xuefei (Rebecca) Yuan >>> Postdoctoral Fellow >>> Lawrence Berkeley National Laboratory >>> Tel: 1-510-486-7031 >>> >>> >>> > > From travis.fisher at nasa.gov Fri Jul 29 11:27:35 2011 From: travis.fisher at nasa.gov (Travis C. Fisher) Date: Fri, 29 Jul 2011 12:27:35 -0400 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: References: Message-ID: <4E32DF77.5010803@nasa.gov> Hong, In petsc-dev, superlu_dist now fails at the first attempt to apply the preconditioner in the first linear solve. In the release version, it worked fine for the first time step (that's five nonlinear solves using the same preconditioner), but then failed on the second time step (which I suppose is more disconcerting). I have tried on 1 and 4 processors and get the same problem in the development version. To clarify, when I mentioned ASM/ILU before, the ILU routine I was using was the default PETSc routine. That still works with my implementation using petsc-dev. Mumps worked fine with petsc-dev, but I don't think it's as fast as superlu_dist for the test problem I am using. Thanks, Travis From bsmith at mcs.anl.gov Fri Jul 29 12:18:31 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jul 2011 12:18:31 -0500 Subject: [petsc-users] force ksp to continue if divergence or increase divergence tol, how ? In-Reply-To: References: Message-ID: <1C560F44-5937-4528-8AA8-8039053E77D8@mcs.anl.gov> On Jul 29, 2011, at 10:10 AM, Pierre-Yves Aquilanti wrote: > Hello, > > I'm using a preconditionner of my construction which has the particularity to make the norm of the residual vector increase suddently (from 1e-1 to 1e+6). The problem is that if that PETSc concludes to divergence after applying my preconditionner. > > Also, i setted the -ksp_divtol flag to 100 (random value to make it works) and also used the KSPSetTolerances function with the same value. 100 is not enough. In your case you need to use something like 10^8. The value you put in this parameter is the maximum ratio the residual norm is allowed to increase before it is declared diverged. Barry > > I'm probably missing some point but can't figure out which one. > > Do someone has an idea ? > > Thanks a lot. > > From bsmith at mcs.anl.gov Fri Jul 29 12:30:23 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jul 2011 12:30:23 -0500 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: <4E32DF77.5010803@nasa.gov> References: <4E32DF77.5010803@nasa.gov> Message-ID: <0A9A2BB9-2D53-42CC-8F4E-CE46E390F3D0@mcs.anl.gov> On Jul 29, 2011, at 11:27 AM, Travis C. Fisher wrote: > Hong, > > In petsc-dev, superlu_dist now fails at the first attempt to apply the preconditioner in the first linear solve. In the release version, it worked fine for the first time step (that's five nonlinear solves using the same preconditioner), but then failed on the second time step (which I suppose is more disconcerting). I have tried on 1 and 4 processors and get the same problem in the development version. Are you absolutely sure the non-zero pattern is fixed and doesn't change? After the call to MatAssemblyEnd() make a call MatSetOption(mat, MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE); Then rerun. Also it is much more useful if you include the entire error message (cut and paste) instead just a few words, there is lots of valuable information in the error message that can help other people see what the solution might be. Barry > > To clarify, when I mentioned ASM/ILU before, the ILU routine I was using was the default PETSc routine. That still works with my implementation using petsc-dev. > > Mumps worked fine with petsc-dev, but I don't think it's as fast as superlu_dist for the test problem I am using. > > Thanks, > > Travis From travis.fisher at nasa.gov Fri Jul 29 12:52:54 2011 From: travis.fisher at nasa.gov (Travis C. Fisher) Date: Fri, 29 Jul 2011 13:52:54 -0400 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: <4E32DF77.5010803@nasa.gov> References: <4E32DF77.5010803@nasa.gov> Message-ID: <4E32F376.4070908@nasa.gov> Barry, I added the line, call MatSetOption(pcmat,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE,ierr), after the call to matrix assembly and do not get an error associated with the nonzero locations. Here is the error running superlu_dist with one processor. 0 SNES Function norm 9.448256037271e+02 Nonzeros in L 21005940 Nonzeros in U 21018240 nonzeros in L+U 41958975 nonzeros in LSUB 3914621 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] MatSolve_SuperLU_DIST line 117 /scratch/tfisher/petsc-dev/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c [0]PETSC ERROR: [0] MatSolve line 3039 /scratch/tfisher/petsc-dev/src/mat/interface/matrix.c [0]PETSC ERROR: [0] PCApply_LU line 202 /scratch/tfisher/petsc-dev/src/ksp/pc/impls/factor/lu/lu.c [0]PETSC ERROR: [0] PCApply line 373 /scratch/tfisher/petsc-dev/src/ksp/pc/interface/precon.c [0]PETSC ERROR: [0] KSPInitialResidual line 45 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itres.c [0]PETSC ERROR: [0] KSPSolve_LGMRES line 354 /scratch/tfisher/petsc-dev/src/ksp/ksp/impls/gmres/lgmres/lgmres.c [0]PETSC ERROR: [0] KSPSolve line 330 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: [0] SNES_KSPSolve line 3364 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c [0]PETSC ERROR: [0] SNESSolve_LS line 142 /scratch/tfisher/petsc-dev/src/snes/impls/ls/ls.c [0]PETSC ERROR: [0] SNESSolve line 2617 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: 77f4001db099d653cddbbaf61d75c0cf8cf833fd HG Date: Fri Jul 29 13:41:12 2011 +0200 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./HFCFS on a arch-linu named larss-nad by tcfishe1 Fri Jul 29 13:47:04 2011 [0]PETSC ERROR: Libraries linked from /ump/fldmd/home/tcfishe1/gnucompiled/lib [0]PETSC ERROR: Configure run at Fri Jul 29 11:37:36 2011 [0]PETSC ERROR: Configure options --prefix=/ump/fldmd/home/tcfishe1/gnucompiled --download-fblas-lapack=yes --download-superlu_dist=yes --download-parmetis=yes --download-hypre=yes --download-mumps=yes --download-blacs=yes --download-scalapack=yes [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 59. Thanks, Travis From bsmith at mcs.anl.gov Fri Jul 29 13:11:33 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jul 2011 13:11:33 -0500 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: <4E32F376.4070908@nasa.gov> References: <4E32DF77.5010803@nasa.gov> <4E32F376.4070908@nasa.gov> Message-ID: Ok, next step is valgrind http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind valgrind is an amazing tool (and I don't say the easily) to help track down memory corruption problems that otherwise are difficult to find. It is likely that valgrind can resolve this issue in a few minutes. Barry On Jul 29, 2011, at 12:52 PM, Travis C. Fisher wrote: > Barry, > > I added the line, > call MatSetOption(pcmat,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE,ierr), > after the call to matrix assembly and do not get an error associated with the nonzero locations. > > Here is the error running superlu_dist with one processor. > > 0 SNES Function norm 9.448256037271e+02 > Nonzeros in L 21005940 > Nonzeros in U 21018240 > nonzeros in L+U 41958975 > nonzeros in LSUB 3914621 > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors > [0]PETSC ERROR: likely location of problem given in stack below > [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, > [0]PETSC ERROR: INSTEAD the line number of the start of the function > [0]PETSC ERROR: is given. > [0]PETSC ERROR: [0] MatSolve_SuperLU_DIST line 117 /scratch/tfisher/petsc-dev/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c > [0]PETSC ERROR: [0] MatSolve line 3039 /scratch/tfisher/petsc-dev/src/mat/interface/matrix.c > [0]PETSC ERROR: [0] PCApply_LU line 202 /scratch/tfisher/petsc-dev/src/ksp/pc/impls/factor/lu/lu.c > [0]PETSC ERROR: [0] PCApply line 373 /scratch/tfisher/petsc-dev/src/ksp/pc/interface/precon.c > [0]PETSC ERROR: [0] KSPInitialResidual line 45 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itres.c > [0]PETSC ERROR: [0] KSPSolve_LGMRES line 354 /scratch/tfisher/petsc-dev/src/ksp/ksp/impls/gmres/lgmres/lgmres.c > [0]PETSC ERROR: [0] KSPSolve line 330 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: [0] SNES_KSPSolve line 3364 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c > [0]PETSC ERROR: [0] SNESSolve_LS line 142 /scratch/tfisher/petsc-dev/src/snes/impls/ls/ls.c > [0]PETSC ERROR: [0] SNESSolve line 2617 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Signal received! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: 77f4001db099d653cddbbaf61d75c0cf8cf833fd HG Date: Fri Jul 29 13:41:12 2011 +0200 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HFCFS on a arch-linu named larss-nad by tcfishe1 Fri Jul 29 13:47:04 2011 > [0]PETSC ERROR: Libraries linked from /ump/fldmd/home/tcfishe1/gnucompiled/lib > [0]PETSC ERROR: Configure run at Fri Jul 29 11:37:36 2011 > [0]PETSC ERROR: Configure options --prefix=/ump/fldmd/home/tcfishe1/gnucompiled --download-fblas-lapack=yes --download-superlu_dist=yes --download-parmetis=yes --download-hypre=yes --download-mumps=yes --download-blacs=yes --download-scalapack=yes > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 59. > > Thanks, > > Travis > From abhyshr at mcs.anl.gov Fri Jul 29 13:13:14 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Fri, 29 Jul 2011 13:13:14 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: <78FAC6C2-F55B-4DA1-B14E-0EEC9E4CF0AF@mcmillan-mcgee.com> Message-ID: <1687855859.173744.1311963194857.JavaMail.root@zimbra.anl.gov> Jon, I am trying to compile petsc with complex data type right now and will try to test the VI solver with complex variables within the next few hours. > My understanding is that comparison operators (such as < and >) are > not defined for complex numbers, so would it be correct to say that > anyone using complex scalars within PETSc will encounter this problem > with SNESVI? In other words, SNESVI inequality constraints can only be > applied to real variables or real-valued functions of complex > variables? For complex scalars, we only compare the real part of the complex variable and bound,i.e., real(xl) <= real(x) <= real(x_u). For your application, this real part comparison is sufficient since the magnitude is saved in the real part. > >> The residuals are set as follows, and the Jacobian entries are set > >> correspondingly: > >> f[V_block] = {function of adjacent V_blocks, V_electrode} > >> f[V_electrode] = 0.0 + 0.0i > >> f[magnitude(V_electrode)] = 0.0 + 0.0i > >> f[magnitude(I_electrode)] = 0.0 + 0.0i Can you give an example of f for all dofs maybe just for one grid point with the following dof names Vb - V_block Ve - V_electrode Vmag - magnitude(V_electrode) Imag - magnitude(I_electrode) > Since I set all the residuals for both magnitude variables to zero, > and each entry in the Jacobian matrix is a partial derivative of a > residual, I believe the Jacobian entries for both magnitudes would all > be zero as well. However, your point made me realize I have no > Jacobian entry for the partial derivative of f[V_block] w.r.t. > V_electrode. Is that needed? Since f[V_block] is a function of V_electrode, you would need to set the partial derivatives of f[V_block] w.r.t V_electrode for correct jacobian evaluation and for having a good newton convergence. How would one enter it with > MatSetValuesStencil? ex28.c, http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/ksp/ksp/examples/tutorials/ex28.c.html, is a petsc example which shows how to use MatSetValuesStencil with multiple degrees of freedom. See the routine ComputeMatrix in the example. My working understanding was that each degree of > freedom conceptually has its own residual and Jacobian, even though > all residuals are contained in one Vec and all Jacobians contained in > one Mat. > Please bare with us as we are trying to figure out whether the issue is with our VI implementation or something in your code is amiss. I appreciate your patience on this matter. Btw, one quick question, are all the problem formulations in your field done using complex variables or by decomposing the complex term into real/imaginary or magnitude/angle? My background is in electrical power grid and we mostly use either real/imaginary part or magnitude/angle as the variables. Shri ----- Original Message ----- > Thanks for your reply, Shri. Responses below: > > On 2011-07-28, at 1:03 PM, Shri wrote: > > > > > ----- Original Message ----- > >> Hi Barry, Shri, > >> > >> I reworked my implementation to include only the time-independent > >> electrical problem, and to make use of SNESVISetVariableBounds(). > >> However, it does not work quite as I expected: Constraints on > >> variables that are functions of other variables have no effect on > >> the > >> variables by which the functions are defined. > >> > >> My constraints are set on the four degrees of freedom as follows: > >> -SNES_VI_INF <= V_block <= SNES_VI_INF > >> -SNES_VI_INF <= V_electrode <= SNES_VI_INF > >> 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) > >> 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) > > > > Note that while V_block and V_electrode and complex quantities,the > > magnitude terms are actually real numbers. What function did you use > > to compute the magnitude? abs()? sqrt of product of complex number > > with its conjugate? > > I used the "0.0 + 0.0i" there to emphasize that I built PETSc with > complex scalars, so all the scalars are stored as complex even if the > imaginary components are always zero. I have been using std::abs() > from the C++ standard library complex template class > (std::complex). I believe the values of type 'double' returned > by std::abs() are implicitly cast to the real part of > std::complex or PetscScalar, though I usually try to cast > explicitly. > > > This seems like an application where the variables are of type > > complex but the constraints need to be set for functions of the > > variables which are actually real numbers. I am not sure whether > > this can be supported. Maybe Barry or others can shed more > > light on this. > > My understanding is that comparison operators (such as < and >) are > not defined for complex numbers, so would it be correct to say that > anyone using complex scalars within PETSc will encounter this problem > with SNESVI? In other words, SNESVI inequality constraints can only be > applied to real variables or real-valued functions of complex > variables? > > >> > >> The residuals are set as follows, and the Jacobian entries are set > >> correspondingly: > >> f[V_block] = {function of adjacent V_blocks, V_electrode} > >> f[V_electrode] = 0.0 + 0.0i > >> f[magnitude(V_electrode)] = 0.0 + 0.0i > >> f[magnitude(I_electrode)] = 0.0 + 0.0i > > > > What equations do you use for the partial derivatives of the > > magnitude functions w.r.t. the actual complex variables (V_block and > > V_electrode) in the jacobian evaluation? > > Since I set all the residuals for both magnitude variables to zero, > and each entry in the Jacobian matrix is a partial derivative of a > residual, I believe the Jacobian entries for both magnitudes would all > be zero as well. However, your point made me realize I have no > Jacobian entry for the partial derivative of f[V_block] w.r.t. > V_electrode. Is that needed? How would one enter it with > MatSetValuesStencil? My working understanding was that each degree of > freedom conceptually has its own residual and Jacobian, even though > all residuals are contained in one Vec and all Jacobians contained in > one Mat. > > >> > >> The first two potentials, V_block and V_electrode, are independent, > >> while magnitude(V_electrode) is a function of V_electrode only and > >> the > >> electrode current magnitude(I_electrode) is a function of V_block > >> and > >> V_electrode. Note that V_electrode acts as a source term in the > >> finite > >> difference formulation for V_block. While all of these variables > >> are > >> complex, the latter two always have zero imaginary parts. I suppose > >> I > >> am assuming that PETSc knows to compare only the real parts if the > >> imaginary parts are zero. (Is this a bad assumption?) > >> > >> When solving with PETSc (using -snes_vi_type rs) > > > > Did you also use -snes_type vi? > > I have been using SNESSetType(snes, SNESVI) in my code, and the use of > -snes_type vi in the command line does not seem to change any > behaviour. > > > > > V_electrode stays at > >> its maximum, as set in the initial guess, and the V_block > >> distribution > >> falls properly from that. However, measurements of > >> magnitude(I_electrode) are way above maximum and do not move > >> downwards > >> with successive SNES iterations. I can also set the constraint on > >> magnitude(V_electrode) to less than maximum and it does not affect > >> the > >> value of V_electrode. How can I tell PETSc to change V_electrode > >> when > >> the magnitude(V_electrode) or magnitude(I_electrode) constraints > >> are > >> not met? > >> > > Send the code for your application with instructions on how to run > > it and we'll try to figure out what needs to be > > done. > > I really appreciate this generous offer of your time, and I will > certainly take you up on it if we cannot resolve this otherwise. > Unfortunately, I will be travelling for the next few weeks and it may > take me some time to isolate the PETSc-dependent parts of my > application. Please bear with me as I will try to keep in touch while > I am away. > > As I mentioned above, I think the issue may lie in the conceptual > incompatibility of complex variables and inequality constraints, but I > would appreciate your thoughts on that. > > Thanks again, > > Jon > > > > > > > Shri > > > > > >> Thanks again for your help. > >> > >> Jon > >> > >> > >> On 2011-07-25, at 8:58 PM, Barry Smith wrote: > >> > >>> > >>> On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > >>> > >>>> Hi Shri, > >>>> > >>>> Thanks for your message and all the helpful tips. If the > >>>> TSVISetVariableBounds() functions are available now, I would like > >>>> to try them as well. Is the interface essentially the same as > >>>> with > >>>> SNESVISetVariableBounds()? I will get back to you and Barry when > >>>> I > >>>> have had a chance to modify my application. > >>>> > >>>> For my problem, I believe it makes sense to have bounds on one of > >>>> the variables as well as one function of the variables. The two > >>>> relevant degrees of freedom are the block voltage (one for each > >>>> finite difference block) and the electrode voltage (one for each > >>>> electrode, which may be present in multiple blocks). The > >>>> electrode > >>>> voltage should keep a constant phase while the magnitude is > >>>> constrained between zero and some maximum. The block voltages > >>>> near > >>>> the electrodes depend on the electrode voltages as well as the > >>>> neighbouring block voltages. The electrode current for a given > >>>> electrode is a function of its electrode voltage and several > >>>> nearby > >>>> block voltages, and should be constrained in magnitude between > >>>> zero > >>>> and some maximum (and the phase unconstrained). Would I need to > >>>> use > >>>> SNESVISetComputeVariableBounds since the electrode current is a > >>>> function of the other variables? Would any other provisions need > >>>> to > >>>> be made for the block voltages, since they depend on the > >>>> electrode > >>>> voltages? > >>>> > >>> > >>> You cannot directly make a bound on some function of other > >>> variables. Instead you need to introduce another variable that is > >>> defined to be equal to that function and put the bound on that > >>> new > >>> variable. > >>> > >>> Barry > >>> > >>>> Thank you again, > >>>> > >>>> Jon > >>>> > >>>> On 2011-07-25, at 11:58 AM, Shri wrote: > >>>> > >>>>> > >>>>> ----- Original Message ----- > >>>>>> On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > >>>>>> > >>>>>>> Barry, > >>>>>>> > >>>>>>> Thank you so much for your response. Lucky, indeed! I look > >>>>>>> forward > >>>>>>> to trying out these new features. > >>>>>>> > >>>>>>> I neglected to mention in my original post that my electrical > >>>>>>> problem is part of a DAE, which includes a time-dependent > >>>>>>> heating > >>>>>>> problem. Can SNESVI constraints be used in conjunction with > >>>>>>> TSSetIFunction() and TSSetIJacobian() as well? (Of course, I > >>>>>>> only > >>>>>>> need the constraints for the time-independent electrical > >>>>>>> portion.) > >>>>>> > >>>>>> We have not yet put that in but Shri is starting to look at > >>>>>> that > >>>>>> just > >>>>>> now. Basically we would have a TSVISetVariableBounds() and > >>>>>> handle > >>>>>> everything from there. I suggest you start with a simple time > >>>>>> electrical portion with constraints to explore and we'll go > >>>>>> from > >>>>>> there. Shri is actually a electrical networks guy and so can > >>>>>> speak > >>>>>> your language. > >>>>> > >>>>> > >>>>> I've added TSVISetVariableBounds() for setting the bounds on the > >>>>> variables using the TS object directly. > >>>>> A few things that i want to mention here about using the > >>>>> variational inequality nonlinear solver (SNESVI). > >>>>> i) Use the runtime option -snes_type vi or explicitly set > >>>>> SNESSetType(snes,SNESVI). > >>>>> ii) There are two tested algorithms currently available, (a) > >>>>> semismooth (-snes_vi_type ss) and (b) active set or reduced > >>>>> space > >>>>> (-snes_vi_type rs). > >>>>> iii) Take a look at example,ex61.c, in > >>>>> src/snes/examples/tutorials > >>>>> which is a time-stepping nonlinear problem with constraints on > >>>>> the > >>>>> variables. This example does not use the TS object directly but > >>>>> rather a time-loop is explicitly written. Another example,ex8.c, > >>>>> in src/snes/examples/tests/ is a minimum surface area problem > >>>>> which uses SNESVI. > >>>>> > >>>>> By the way, does your problem have bounds on the variables or > >>>>> bounds on some function of the variables? > >>>>> > >>>>> Shri > >>>>> > >>>>> > >>>>> > >>>>>> > >>>>>> Barry > >>>>>> > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> Thank you again, > >>>>>>> > >>>>>>> Jon > >>>>>>> > >>>>>>> On 2011-07-22, at 2:46 PM, Barry Smith wrote: > >>>>>>> > >>>>>>>> > >>>>>>>> Jon, > >>>>>>>> > >>>>>>>> You may be in luck. In PETSc-dev > >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > >>>>>>>> we > >>>>>>>> have now implemented variational inequality non-linear > >>>>>>>> solvers > >>>>>>>> with box constraints. > >>>>>>>> > >>>>>>>> That is one has a set of variables u and algebraic equations > >>>>>>>> F(u) = 0 (say of size n each) but in addition one has > >>>>>>>> constraints lu <= u <= uu where some or all of lu may be > >>>>>>>> negative infinity (no constraints) and some or all of uu may > >>>>>>>> be > >>>>>>>> infinity (no constraints). There is also a constraint on the > >>>>>>>> sign of F() for those equations associated with active > >>>>>>>> constraints. If your constraints are not in this form > >>>>>>>> sometimes > >>>>>>>> you can introduce new additional variables to get it in this > >>>>>>>> form. Read up a little on variational inequalities on the > >>>>>>>> web. > >>>>>>>> > >>>>>>>> To use this you provide the usual SNES function and Jacobian > >>>>>>>> but > >>>>>>>> you also provide SNESVISetVariableBounds() there is a manual > >>>>>>>> page for this function and for for SNESVI at > >>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html > >>>>>>>> (the link is broken right now but Satish is fixing it). Plus > >>>>>>>> several examples in src/snes/examples/tutorials (in > >>>>>>>> petsc-dev). > >>>>>>>> > >>>>>>>> This is all new code so we would be interested in your > >>>>>>>> feedback. > >>>>>>>> > >>>>>>>> > >>>>>>>> Barry > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > >>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> I am trying to add a constraint feature to my first PETSc > >>>>>>>>> application, which uses the finite difference method to > >>>>>>>>> calculate > >>>>>>>>> the potential distribution produced by a collection of > >>>>>>>>> electrodes > >>>>>>>>> in a resistive medium. I would like to make this simulation > >>>>>>>>> more > >>>>>>>>> realistic by imposing a maximum electric current and a > >>>>>>>>> maximum > >>>>>>>>> potential difference that can be supplied to each electrode > >>>>>>>>> by > >>>>>>>>> the > >>>>>>>>> power supply. If the medium between the electrodes is very > >>>>>>>>> conductive, the current maximum would be exceeded by the > >>>>>>>>> maximum > >>>>>>>>> potential difference, so the potential difference should be > >>>>>>>>> decreased from maximum until it produces the maximum > >>>>>>>>> current. > >>>>>>>>> On > >>>>>>>>> the other hand, the potential difference between the > >>>>>>>>> electrodes > >>>>>>>>> should remain at maximum as long as the current remains > >>>>>>>>> below > >>>>>>>>> maximum (say, for a less conductive medium). > >>>>>>>>> > >>>>>>>>> I added an extra degree of freedom (the electrode voltages) > >>>>>>>>> to > >>>>>>>>> my > >>>>>>>>> DMDA, and I developed a set of conditional expressions that > >>>>>>>>> describe the above constraints, but one problem is that the > >>>>>>>>> logic > >>>>>>>>> relies on if-then-else decisions that are made when forming > >>>>>>>>> the > >>>>>>>>> function/residual and the Jacobian. Once these decisions are > >>>>>>>>> made, > >>>>>>>>> of course, the conditions are not checked again until the > >>>>>>>>> next > >>>>>>>>> function or Jacobian evaluation. The non-linear solver then > >>>>>>>>> tends > >>>>>>>>> to oscillate between extreme solutions to the opposing > >>>>>>>>> conditions > >>>>>>>>> with each iteration, and never converges towards a > >>>>>>>>> reasonable > >>>>>>>>> solution. > >>>>>>>>> > >>>>>>>>> Is there a better strategy for solving such problems? Does > >>>>>>>>> PETSc > >>>>>>>>> offer mechanisms to aid in their solution? I would very much > >>>>>>>>> appreciate any hints. > >>>>>>>>> > >>>>>>>>> Thank you for your time, > >>>>>>>>> > >>>>>>>>> Jon > >>>>>>>> > >>>>>>> > >>>>> > >>>> > >>> > > From hzhang at mcs.anl.gov Fri Jul 29 13:30:55 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 29 Jul 2011 13:30:55 -0500 Subject: [petsc-users] Preconditioned JFNK using SuperLU_dist for time dependent problems In-Reply-To: References: <4E32DF77.5010803@nasa.gov> <4E32F376.4070908@nasa.gov> Message-ID: Have you tried options '-mat_superlu_dist_equil NO -mat_superlu_dist_rowperm NATURAL -mat_superlu_dist_colperm NATURAL '? In addition, how do you know mumps is not as fast as superlu_dist? Hong On Fri, Jul 29, 2011 at 1:11 PM, Barry Smith wrote: > > ?Ok, next step is valgrind http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind > > ? valgrind is an amazing tool (and I don't say the easily) to help track down memory corruption problems that otherwise are difficult to find. ?It is likely that valgrind can resolve this issue in a few minutes. > > ? Barry > > > > On Jul 29, 2011, at 12:52 PM, Travis C. Fisher wrote: > >> Barry, >> >> I added the line, >> call MatSetOption(pcmat,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE,ierr), >> after the call to matrix assembly and do not get an error associated with the nonzero locations. >> >> Here is the error running superlu_dist with one processor. >> >> 0 SNES Function norm 9.448256037271e+02 >> ? ? ? Nonzeros in L ? ? ? 21005940 >> ? ? ? Nonzeros in U ? ? ? 21018240 >> ? ? ? nonzeros in L+U ? ? 41958975 >> ? ? ? nonzeros in LSUB ? ?3914621 >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range >> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger >> [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors >> [0]PETSC ERROR: likely location of problem given in stack below >> [0]PETSC ERROR: --------------------- ?Stack Frames ------------------------------------ >> [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, >> [0]PETSC ERROR: ? ? ? INSTEAD the line number of the start of the function >> [0]PETSC ERROR: ? ? ? is given. >> [0]PETSC ERROR: [0] MatSolve_SuperLU_DIST line 117 /scratch/tfisher/petsc-dev/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c >> [0]PETSC ERROR: [0] MatSolve line 3039 /scratch/tfisher/petsc-dev/src/mat/interface/matrix.c >> [0]PETSC ERROR: [0] PCApply_LU line 202 /scratch/tfisher/petsc-dev/src/ksp/pc/impls/factor/lu/lu.c >> [0]PETSC ERROR: [0] PCApply line 373 /scratch/tfisher/petsc-dev/src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: [0] KSPInitialResidual line 45 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itres.c >> [0]PETSC ERROR: [0] KSPSolve_LGMRES line 354 /scratch/tfisher/petsc-dev/src/ksp/ksp/impls/gmres/lgmres/lgmres.c >> [0]PETSC ERROR: [0] KSPSolve line 330 /scratch/tfisher/petsc-dev/src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: [0] SNES_KSPSolve line 3364 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c >> [0]PETSC ERROR: [0] SNESSolve_LS line 142 /scratch/tfisher/petsc-dev/src/snes/impls/ls/ls.c >> [0]PETSC ERROR: [0] SNESSolve line 2617 /scratch/tfisher/petsc-dev/src/snes/interface/snes.c >> [0]PETSC ERROR: --------------------- Error Message ------------------------------------ >> [0]PETSC ERROR: Signal received! >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Development HG revision: 77f4001db099d653cddbbaf61d75c0cf8cf833fd ?HG Date: Fri Jul 29 13:41:12 2011 +0200 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HFCFS on a arch-linu named larss-nad by tcfishe1 Fri Jul 29 13:47:04 2011 >> [0]PETSC ERROR: Libraries linked from /ump/fldmd/home/tcfishe1/gnucompiled/lib >> [0]PETSC ERROR: Configure run at Fri Jul 29 11:37:36 2011 >> [0]PETSC ERROR: Configure options --prefix=/ump/fldmd/home/tcfishe1/gnucompiled --download-fblas-lapack=yes --download-superlu_dist=yes --download-parmetis=yes --download-hypre=yes --download-mumps=yes --download-blacs=yes --download-scalapack=yes >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file >> -------------------------------------------------------------------------- >> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD >> with errorcode 59. >> >> Thanks, >> >> Travis >> > > From zhenglun.wei at gmail.com Fri Jul 29 14:51:18 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Fri, 29 Jul 2011 14:51:18 -0500 Subject: [petsc-users] Question on re-coding ex29 In-Reply-To: References: Message-ID: Problem resolved. Thank you so much, John. best, Alan On Thu, Jul 28, 2011 at 1:42 PM, John Chludzinski wrote: > In ComputeRHS.h use: > > #ifndef _COMPURERHS_H > #define _COMPURERHS_H > > PetscErrorCode ComputeRHS(DMMG dmmg, Vec b); > > #endif > > ---John > > > On Thu, Jul 28, 2011 at 11:22 AM, Alan Wei wrote: > >> Thanks, John. However, on my opinion, one of them has been commented out. >> ^_^ >> >> best, >> Alan >> >> >> On Thu, Jul 28, 2011 at 12:55 AM, John Chludzinski < >> jchludzinski at gmail.com> wrote: >> >>> You have: >>> >>> *typedef enum {DIRICHLET, NEUMANN} BCType;* >>> >>> twice in your source: ex29.c >>> >>> ---John >>> >>> >>> On Wed, Jul 27, 2011 at 3:46 PM, Alan Wei >>> wrote: >>> > Dear John, >>> > Thanks for your quick reply. I think the error message says that >>> the >>> > BCType has some problem. However, it is originally there and I have not >>> > changed it at all. Is that because I define it in ComputeRHS again and >>> let >>> > ex29.c include ComputeRHS.h so the BCtype is duplicated? >>> > >>> > thanks, >>> > Alan >>> > >>> > On Wed, Jul 27, 2011 at 2:04 PM, John Chludzinski < >>> jchludzinski at gmail.com> >>> > wrote: >>> >> >>> >> Without wading through the attached zip file, it looks like you're >>> missing >>> >> some typedef's that come from a missing include file. ---John >>> >> >>> >> On Wed, Jul 27, 2011 at 2:38 PM, Alan Wei >>> wrote: >>> >>> >>> >>> Dear Sir/Madam, >>> >>> I hope you're having a nice day. >>> >>> I asked several questions about >>> src/ksp/ksp/example/tutorial/ex29.c >>> >>> before, and learned some beginning staff of PETSc. Right now, I want >>> to >>> >>> start to use it to my own program. Firstly, I want to isolate the >>> ComputeRHS >>> >>> function in ex29, which means I want to put ComputeRHS to another c >>> program >>> >>> file. I did that; however, some problem pop up when I compile it. >>> >>> >>> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >>> >>> ex29.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>> -Wno-unknown-pragmas >>> >>> -g3 -I/home/zlwei/soft/mercurial/petsc-dev/include >>> >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >>> >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >>> >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ex29.c >>> >>> ex29.c: In function ComputeRHS: >>> >>> ex29.c:39: error: storage class specified for parameter BCType >>> >>> ex29.c:44: error: expected specifier-qualifier-list before BCType >>> >>> ex29.c:61: error: expected =, ,, ;, asm or __attribute__ before { >>> token >>> >>> ex29.c:117: error: expected =, ,, ;, asm or __attribute__ before { >>> token >>> >>> ex29.c:130: error: expected =, ,, ;, asm or __attribute__ before { >>> token >>> >>> ex29.c:197: error: expected =, ,, ;, asm or __attribute__ before { >>> token >>> >>> ex29.c:220: error: old-style parameter declarations in prototyped >>> >>> function definition >>> >>> ex29.c:220: error: expected { at end of input >>> >>> make: [ex29.o] Error 1 (ignored) >>> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc -o >>> >>> ComputeRHS.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>> >>> -Wno-unknown-pragmas -g3 >>> -I/home/zlwei/soft/mercurial/petsc-dev/include >>> >>> -I/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/include >>> >>> -I/home/zlwei/soft/mercurial/petsc-dev/src/dm/mesh/sieve >>> >>> -D__INSDIR__=src/ksp/ksp/examples/tutorials/ ComputeRHS.c >>> >>> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/bin/mpicc >>> -Wall >>> >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -o >>> ex29 >>> >>> ex29.o ComputeRHS.o >>> >>> -L/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >>> -lpetsc >>> >>> >>> -Wl,-rpath,/home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib >>> >>> -lflapack -lfblas -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -ldl >>> -lmpich >>> >>> -lopa -lmpl -lrt -lpthread -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl >>> >>> -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl >>> >>> gcc: ex29.o: No such file or directory >>> >>> make: [ex29] Error 1 (ignored) >>> >>> >>> >>> The program and modified makefile are attached. Could you please take >>> a >>> >>> look and give me some suggestions. >>> >>> >>> >>> best, >>> >>> Alan >>> >> >>> > >>> > >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Fri Jul 29 14:52:33 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Fri, 29 Jul 2011 12:52:33 -0700 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: Matt, Is ParMetis implementation in PETSc only done via $PETSC_DIR/src/mat/partition/impls/pmetis/pmetis.c ? I am wondering if PETSc has interface to ParMETIS_V3_RefineKway function as noted in ParMetis manual? Thanks, Mohammad On Thu, Jul 28, 2011 at 10:11 PM, Mohammad Mirzadeh wrote: > I see. Thanks for the help Matt > > > On Thu, Jul 28, 2011 at 10:01 PM, Matthew Knepley wrote: > >> On Fri, Jul 29, 2011 at 4:52 AM, Mohammad Mirzadeh wrote: >> >>> Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est >>> uses parallel octrees/quadtrees but for what I intend to do I only need to >>> distribute a single tree that is created in serial among processors. >>> I definitely like to have the tree data-structure in parallel but that would >>> be another project. I also looked into Dendro and they kind of follow the >>> same strategy. i.e every single processor has a local copy of the whole >>> tree. What they do differently, however, is they somehow manage to use DA >>> instead of a general unstructured numbering which is quite interesting but I >>> still don't know how they do it. Unfortunately, they do not handle (as far >>> as I understood from their manual) non-graded trees which are the ones I >>> work with. >>> >>> So, all I need to do is to somehow distribute my grid among processors >>> and since each one has a local copy of data-structure I could get around the >>> problem. Just anotehr question. If the partitioning is not unique, do you at >>> least get a better numbering than the tree you start with? >>> >> >> You should, which is why I suggested that you are not giving the input you >> think you are. >> >> Matt >> >> >>> Mohammad >>> >>> >>> On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: >>> >>>> On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh wrote: >>>> >>>>> Hi all, >>>>> >>>>> I am trying to write a code to do parallel computation on quadtree >>>>> adaptive grids and to do so , I need to distribute the grid in parallel. I >>>>> have selected a general unstructured framework for telling PETSc about my >>>>> node numbering. An example of such grid is schematically shown below. >>>>> >>>> >>>> 0) If you are doing this, I think you should at least look at the p4est >>>> package before proceeding. >>>> >>>> >>>>> >>>>> 1 16 7 3 >>>>> +---------------+---------------+------------------------------+ >>>>> | | | | >>>>> | | | | >>>>> |14 | 15 | 17 | >>>>> +---------------+---------------+ | >>>>> | | | | >>>>> | | | | >>>>> | 4 | 12 | 6 |8 >>>>> +---------------+---------------+------------------------------+ >>>>> | | | | >>>>> | | | | >>>>> | 9 | 11 | 13 | >>>>> +---------------+---------------+ | >>>>> | | | | >>>>> | | | | >>>>> | 0 | 10 |5 | 2 >>>>> +---------------+---------------+------------------------------+ >>>>> >>>>> >>>>> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >>>>> >>>>> >>>>> Number of elements in ordering 18 >>>>> PETSc->App App->PETSc >>>>> 0 9 0 1 >>>>> 1 0 1 3 >>>>> 2 10 2 4 >>>>> 3 1 3 7 >>>>> 4 2 4 12 >>>>> 5 11 5 14 >>>>> 6 12 6 15 >>>>> 7 3 7 16 >>>>> 8 13 8 17 >>>>> 9 14 9 0 >>>>> 10 15 10 2 >>>>> 11 16 11 5 >>>>> 12 4 12 6 >>>>> 13 17 13 8 >>>>> 14 5 14 9 >>>>> 15 6 15 10 >>>>> 16 7 16 11 >>>>> 17 8 17 13 >>>>> >>>>> Now I have two questions/concerns: >>>>> >>>>> 1) Do processors always have the nodes in contiguous chunks of PETSc >>>>> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >>>>> ordering does not seem to be "good" for this grid since it seems to cross >>>>> too many edges in the graph (here 13 edges) and by just looking at the graph >>>>> I can(at least) think of a better distribution with only 6 edge cuts. (if >>>>> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >>>>> rank 1). >>>>> >>>> >>>> Yes, the PETSc ordering is always contiguous. Perhaps you are not >>>> providing the graph you think you are for partitioning. >>>> >>>> >>>>> 2) Isn't it true that the final distribution should be independent of >>>>> initial grid numbering? When I try the same grid but with the following >>>>> (hand-generated) numbering: >>>>> >>>>> 14 15 16 17 >>>>> +---------------+---------------+------------------------------+ >>>>> | | | | >>>>> | | | | >>>>> |11 | 12 | 13 | >>>>> +---------------+---------------+ | >>>>> | | | | >>>>> | | | | >>>>> | 7 | 8 | 9 |10 >>>>> +---------------+---------------+------------------------------+ >>>>> | | | | >>>>> | | | | >>>>> | 4 | 5 | 6 | >>>>> +---------------+---------------+ | >>>>> | | | | >>>>> | | | | >>>>> | 0 | 1 |2 | 3 >>>>> +---------------+---------------+------------------------------+ >>>>> >>>>> I get the following AO: >>>>> >>>>> Number of elements in ordering 18 >>>>> PETSc->App App->PETSc >>>>> 0 9 0 9 >>>>> 1 10 1 10 >>>>> 2 11 2 11 >>>>> 3 12 3 12 >>>>> 4 13 4 13 >>>>> 5 14 5 14 >>>>> 6 15 6 15 >>>>> 7 16 7 16 >>>>> 8 17 8 17 >>>>> 9 0 9 0 >>>>> 10 1 10 1 >>>>> 11 2 11 2 >>>>> 12 3 12 3 >>>>> 13 4 13 4 >>>>> 14 5 14 5 >>>>> 15 6 15 6 >>>>> 16 7 16 7 >>>>> 17 8 17 8 >>>>> >>>>> >>>>> which is simply the initial ordering with a change in the order in >>>>> which processors handle nodes. Could it be that the partitioning is not >>>>> unique and each time the algorithm only tries to obtain the "best" possible >>>>> ordering depending on the initial distribution? If so, how should I know >>>>> what ordering to start with? >>>>> >>>> >>>> Yes, ParMetis does not provide a unique "best" ordering, which is at >>>> least NP-complete if not worse. >>>> >>>> Matt >>>> >>>> >>>>> I am really confused and would appreciate if someone could provide some >>>>> insights. >>>>> >>>>> Thanks, >>>>> Mohammad >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Jul 29 16:09:21 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 29 Jul 2011 21:09:21 +0000 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 7:52 PM, Mohammad Mirzadeh wrote: > Matt, > > Is ParMetis implementation in PETSc only done via > $PETSC_DIR/src/mat/partition/impls/pmetis/pmetis.c ? I am wondering if PETSc > has interface to ParMETIS_V3_RefineKway function as noted in ParMetis > manual? > If you want to Mat, then pmetis.c is it. There is stuff for more general meshes, but it is experimental. There is a paper about it http://arxiv.org/abs/0908.4427 which will give you an idea what it does. Thanks, Matt > Thanks, > Mohammad > > > On Thu, Jul 28, 2011 at 10:11 PM, Mohammad Mirzadeh wrote: > >> I see. Thanks for the help Matt >> >> >> On Thu, Jul 28, 2011 at 10:01 PM, Matthew Knepley wrote: >> >>> On Fri, Jul 29, 2011 at 4:52 AM, Mohammad Mirzadeh wrote: >>> >>>> Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est >>>> uses parallel octrees/quadtrees but for what I intend to do I only need to >>>> distribute a single tree that is created in serial among processors. >>>> I definitely like to have the tree data-structure in parallel but that would >>>> be another project. I also looked into Dendro and they kind of follow the >>>> same strategy. i.e every single processor has a local copy of the whole >>>> tree. What they do differently, however, is they somehow manage to use DA >>>> instead of a general unstructured numbering which is quite interesting but I >>>> still don't know how they do it. Unfortunately, they do not handle (as far >>>> as I understood from their manual) non-graded trees which are the ones I >>>> work with. >>>> >>>> So, all I need to do is to somehow distribute my grid among processors >>>> and since each one has a local copy of data-structure I could get around the >>>> problem. Just anotehr question. If the partitioning is not unique, do you at >>>> least get a better numbering than the tree you start with? >>>> >>> >>> You should, which is why I suggested that you are not giving the input >>> you think you are. >>> >>> Matt >>> >>> >>>> Mohammad >>>> >>>> >>>> On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: >>>> >>>>> On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh >>>> > wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I am trying to write a code to do parallel computation on quadtree >>>>>> adaptive grids and to do so , I need to distribute the grid in parallel. I >>>>>> have selected a general unstructured framework for telling PETSc about my >>>>>> node numbering. An example of such grid is schematically shown below. >>>>>> >>>>> >>>>> 0) If you are doing this, I think you should at least look at the p4est >>>>> package before proceeding. >>>>> >>>>> >>>>>> >>>>>> 1 16 7 3 >>>>>> +---------------+---------------+------------------------------+ >>>>>> | | | | >>>>>> | | | | >>>>>> |14 | 15 | 17 | >>>>>> +---------------+---------------+ | >>>>>> | | | | >>>>>> | | | | >>>>>> | 4 | 12 | 6 |8 >>>>>> +---------------+---------------+------------------------------+ >>>>>> | | | | >>>>>> | | | | >>>>>> | 9 | 11 | 13 | >>>>>> +---------------+---------------+ | >>>>>> | | | | >>>>>> | | | | >>>>>> | 0 | 10 |5 | 2 >>>>>> +---------------+---------------+------------------------------+ >>>>>> >>>>>> >>>>>> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >>>>>> >>>>>> >>>>>> Number of elements in ordering 18 >>>>>> PETSc->App App->PETSc >>>>>> 0 9 0 1 >>>>>> 1 0 1 3 >>>>>> 2 10 2 4 >>>>>> 3 1 3 7 >>>>>> 4 2 4 12 >>>>>> 5 11 5 14 >>>>>> 6 12 6 15 >>>>>> 7 3 7 16 >>>>>> 8 13 8 17 >>>>>> 9 14 9 0 >>>>>> 10 15 10 2 >>>>>> 11 16 11 5 >>>>>> 12 4 12 6 >>>>>> 13 17 13 8 >>>>>> 14 5 14 9 >>>>>> 15 6 15 10 >>>>>> 16 7 16 11 >>>>>> 17 8 17 13 >>>>>> >>>>>> Now I have two questions/concerns: >>>>>> >>>>>> 1) Do processors always have the nodes in contiguous chunks of PETSc >>>>>> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >>>>>> ordering does not seem to be "good" for this grid since it seems to cross >>>>>> too many edges in the graph (here 13 edges) and by just looking at the graph >>>>>> I can(at least) think of a better distribution with only 6 edge cuts. (if >>>>>> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >>>>>> rank 1). >>>>>> >>>>> >>>>> Yes, the PETSc ordering is always contiguous. Perhaps you are not >>>>> providing the graph you think you are for partitioning. >>>>> >>>>> >>>>>> 2) Isn't it true that the final distribution should be independent of >>>>>> initial grid numbering? When I try the same grid but with the following >>>>>> (hand-generated) numbering: >>>>>> >>>>>> 14 15 16 17 >>>>>> +---------------+---------------+------------------------------+ >>>>>> | | | | >>>>>> | | | | >>>>>> |11 | 12 | 13 | >>>>>> +---------------+---------------+ | >>>>>> | | | | >>>>>> | | | | >>>>>> | 7 | 8 | 9 |10 >>>>>> +---------------+---------------+------------------------------+ >>>>>> | | | | >>>>>> | | | | >>>>>> | 4 | 5 | 6 | >>>>>> +---------------+---------------+ | >>>>>> | | | | >>>>>> | | | | >>>>>> | 0 | 1 |2 | 3 >>>>>> +---------------+---------------+------------------------------+ >>>>>> >>>>>> I get the following AO: >>>>>> >>>>>> Number of elements in ordering 18 >>>>>> PETSc->App App->PETSc >>>>>> 0 9 0 9 >>>>>> 1 10 1 10 >>>>>> 2 11 2 11 >>>>>> 3 12 3 12 >>>>>> 4 13 4 13 >>>>>> 5 14 5 14 >>>>>> 6 15 6 15 >>>>>> 7 16 7 16 >>>>>> 8 17 8 17 >>>>>> 9 0 9 0 >>>>>> 10 1 10 1 >>>>>> 11 2 11 2 >>>>>> 12 3 12 3 >>>>>> 13 4 13 4 >>>>>> 14 5 14 5 >>>>>> 15 6 15 6 >>>>>> 16 7 16 7 >>>>>> 17 8 17 8 >>>>>> >>>>>> >>>>>> which is simply the initial ordering with a change in the order in >>>>>> which processors handle nodes. Could it be that the partitioning is not >>>>>> unique and each time the algorithm only tries to obtain the "best" possible >>>>>> ordering depending on the initial distribution? If so, how should I know >>>>>> what ordering to start with? >>>>>> >>>>> >>>>> Yes, ParMetis does not provide a unique "best" ordering, which is at >>>>> least NP-complete if not worse. >>>>> >>>>> Matt >>>>> >>>>> >>>>>> I am really confused and would appreciate if someone could provide >>>>>> some insights. >>>>>> >>>>>> Thanks, >>>>>> Mohammad >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Jul 29 16:40:51 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 29 Jul 2011 16:40:51 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 23:40, Vijay S. Mahadevan wrote: > I was also going to ask whether this can be used to solve a DAE by > decoupling it into a time dependent equation and an algebraic (functional) > constraint associated with it. The stiff integrators in TS are intended for solving DAE. If you have consistent initial conditions, then TSGL should just work, perhaps after turning off error control or specifying your own controller (the built-in controllers are not robust at present). If you do not have consistent initial conditions, then they need to be solved for, which we don't currently have support for, but should be adding reasonably soon. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Fri Jul 29 16:58:15 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Fri, 29 Jul 2011 14:58:15 -0700 Subject: [petsc-users] Grid Partitioning with ParMetis In-Reply-To: References: Message-ID: Thanks Matt for your help and support. I read a little on ParMetis manual and looked at PETSc implementation and I think I figured out the source of my confusion. The fact is, ParMetis does create a good partitioning starting from the initial numbering. Indeed according to their manual for ParMetis 3.2 and on page 5: "... ParMETIS_V3_PartKway makes no assumptions on how the graph is initially distributed among the processors. It can effectively partition a graph that is randomly distributed as well as a graph that is well distributed . If the graph is initially well distributed among the processors, ParMETIS_V3_PartKway will take less time to run. However, the quality of the computed partitionings does not depend on the initial distribution." The problem that I was having was looking at the final AO and assuming that nodes 0-8 are on rank 0 and 9-17 on rank 1 of the *new partitioned grid. *This is wrong since "ISPartitioningToNumbering()" generates an index set "is" such that "on each processor the index set that defines the global numbers (in the new numbering) for all the nodes currently (before the partitioning) on that processor". In other words, I still need to change the numbering such that each processors has the newly computed numbers! This whole thing is little bit too confusing! Anyway, I think the problem is solved now. Thanks again for the help, Mohammad On Fri, Jul 29, 2011 at 2:09 PM, Matthew Knepley wrote: > On Fri, Jul 29, 2011 at 7:52 PM, Mohammad Mirzadeh wrote: > >> Matt, >> >> Is ParMetis implementation in PETSc only done via >> $PETSC_DIR/src/mat/partition/impls/pmetis/pmetis.c ? I am wondering if PETSc >> has interface to ParMETIS_V3_RefineKway function as noted in ParMetis >> manual? >> > > If you want to Mat, then pmetis.c is it. There is stuff for more general > meshes, but it is experimental. There is a paper about it > > http://arxiv.org/abs/0908.4427 > > which will give you an idea what it does. > > Thanks, > > Matt > > >> Thanks, >> Mohammad >> >> >> On Thu, Jul 28, 2011 at 10:11 PM, Mohammad Mirzadeh wrote: >> >>> I see. Thanks for the help Matt >>> >>> >>> On Thu, Jul 28, 2011 at 10:01 PM, Matthew Knepley wrote: >>> >>>> On Fri, Jul 29, 2011 at 4:52 AM, Mohammad Mirzadeh wrote: >>>> >>>>> Thank you Matt. Indeed I have looked into p4est and also Dendro. p4est >>>>> uses parallel octrees/quadtrees but for what I intend to do I only need to >>>>> distribute a single tree that is created in serial among processors. >>>>> I definitely like to have the tree data-structure in parallel but that would >>>>> be another project. I also looked into Dendro and they kind of follow the >>>>> same strategy. i.e every single processor has a local copy of the whole >>>>> tree. What they do differently, however, is they somehow manage to use DA >>>>> instead of a general unstructured numbering which is quite interesting but I >>>>> still don't know how they do it. Unfortunately, they do not handle (as far >>>>> as I understood from their manual) non-graded trees which are the ones I >>>>> work with. >>>>> >>>>> So, all I need to do is to somehow distribute my grid among processors >>>>> and since each one has a local copy of data-structure I could get around the >>>>> problem. Just anotehr question. If the partitioning is not unique, do you at >>>>> least get a better numbering than the tree you start with? >>>>> >>>> >>>> You should, which is why I suggested that you are not giving the input >>>> you think you are. >>>> >>>> Matt >>>> >>>> >>>>> Mohammad >>>>> >>>>> >>>>> On Thu, Jul 28, 2011 at 9:25 PM, Matthew Knepley wrote: >>>>> >>>>>> On Fri, Jul 29, 2011 at 3:49 AM, Mohammad Mirzadeh < >>>>>> mirzadeh at gmail.com> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I am trying to write a code to do parallel computation on quadtree >>>>>>> adaptive grids and to do so , I need to distribute the grid in parallel. I >>>>>>> have selected a general unstructured framework for telling PETSc about my >>>>>>> node numbering. An example of such grid is schematically shown below. >>>>>>> >>>>>> >>>>>> 0) If you are doing this, I think you should at least look at the >>>>>> p4est package before proceeding. >>>>>> >>>>>> >>>>>>> >>>>>>> 1 16 7 3 >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> | | | | >>>>>>> | | | | >>>>>>> |14 | 15 | 17 | >>>>>>> +---------------+---------------+ | >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 4 | 12 | 6 |8 >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 9 | 11 | 13 | >>>>>>> +---------------+---------------+ | >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 0 | 10 |5 | 2 >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> >>>>>>> >>>>>>> To distribute this in parallel I am using the ParMetis interface via MatPartitioning object and I follow(more or less) the example in $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c; To make the initial distribution, I choose nodal based partitioning by creating the adjacency matrix, for which I create ia and ja arrays accordingly. once the grid is processed and the new orderings are generated, I follow all required steps to generate the AO needed to map between PETSc ordering and the new global numbering and this is the result: >>>>>>> >>>>>>> >>>>>>> Number of elements in ordering 18 >>>>>>> PETSc->App App->PETSc >>>>>>> 0 9 0 1 >>>>>>> 1 0 1 3 >>>>>>> 2 10 2 4 >>>>>>> 3 1 3 7 >>>>>>> 4 2 4 12 >>>>>>> 5 11 5 14 >>>>>>> 6 12 6 15 >>>>>>> 7 3 7 16 >>>>>>> 8 13 8 17 >>>>>>> 9 14 9 0 >>>>>>> 10 15 10 2 >>>>>>> 11 16 11 5 >>>>>>> 12 4 12 6 >>>>>>> 13 17 13 8 >>>>>>> 14 5 14 9 >>>>>>> 15 6 15 10 >>>>>>> 16 7 16 11 >>>>>>> 17 8 17 13 >>>>>>> >>>>>>> Now I have two questions/concerns: >>>>>>> >>>>>>> 1) Do processors always have the nodes in contiguous chunks of PETSc >>>>>>> ordering? i.e 0-8 on rank 0 and 9-17 on rank 1 ? If so, this particular >>>>>>> ordering does not seem to be "good" for this grid since it seems to cross >>>>>>> too many edges in the graph (here 13 edges) and by just looking at the graph >>>>>>> I can(at least) think of a better distribution with only 6 edge cuts. (if >>>>>>> you are wondering how, having {0,9,4,14,1,10,11,12,15} on rank 0 and rest on >>>>>>> rank 1). >>>>>>> >>>>>> >>>>>> Yes, the PETSc ordering is always contiguous. Perhaps you are not >>>>>> providing the graph you think you are for partitioning. >>>>>> >>>>>> >>>>>>> 2) Isn't it true that the final distribution should >>>>>>> be independent of initial grid numbering? When I try the same grid but with >>>>>>> the following (hand-generated) numbering: >>>>>>> >>>>>>> 14 15 16 17 >>>>>>> >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> | | | | >>>>>>> | | | | >>>>>>> |11 | 12 | 13 | >>>>>>> +---------------+---------------+ | >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 7 | 8 | 9 |10 >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 4 | 5 | 6 | >>>>>>> +---------------+---------------+ | >>>>>>> | | | | >>>>>>> | | | | >>>>>>> | 0 | 1 |2 | 3 >>>>>>> +---------------+---------------+------------------------------+ >>>>>>> >>>>>>> I get the following AO: >>>>>>> >>>>>>> Number of elements in ordering 18 >>>>>>> PETSc->App App->PETSc >>>>>>> 0 9 0 9 >>>>>>> 1 10 1 10 >>>>>>> 2 11 2 11 >>>>>>> 3 12 3 12 >>>>>>> 4 13 4 13 >>>>>>> 5 14 5 14 >>>>>>> 6 15 6 15 >>>>>>> 7 16 7 16 >>>>>>> 8 17 8 17 >>>>>>> 9 0 9 0 >>>>>>> 10 1 10 1 >>>>>>> 11 2 11 2 >>>>>>> 12 3 12 3 >>>>>>> 13 4 13 4 >>>>>>> 14 5 14 5 >>>>>>> 15 6 15 6 >>>>>>> 16 7 16 7 >>>>>>> 17 8 17 8 >>>>>>> >>>>>>> >>>>>>> which is simply the initial ordering with a change in the order in >>>>>>> which processors handle nodes. Could it be that the partitioning is not >>>>>>> unique and each time the algorithm only tries to obtain the "best" possible >>>>>>> ordering depending on the initial distribution? If so, how should I know >>>>>>> what ordering to start with? >>>>>>> >>>>>> >>>>>> Yes, ParMetis does not provide a unique "best" ordering, which is at >>>>>> least NP-complete if not worse. >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> I am really confused and would appreciate if someone could provide >>>>>>> some insights. >>>>>>> >>>>>>> Thanks, >>>>>>> Mohammad >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Jul 29 18:22:12 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 29 Jul 2011 18:22:12 -0500 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: <4E2F2950.2030701@tuhh.de> References: <4E2F2950.2030701@tuhh.de> Message-ID: On Tue, Jul 26, 2011 at 15:53, Ping Rong wrote: > I have then compiled the current dev-version. but the problem is my program > is based on another library (libMesh) which uses petsc as a solver package. > Since some of the interfaces have been changed in the dev-version, I would > not be able to compile the libMesh with petsc anymore. Are you using the development version of libMesh? I built libMesh with petsc-dev not long ago, and I send updates when I notice changes in petsc-dev that affect libMesh. Is it not working now? -------------- next part -------------- An HTML attachment was scrubbed... URL: From shitij.cse at gmail.com Sat Jul 30 06:11:58 2011 From: shitij.cse at gmail.com (Shitij Bhargava) Date: Sat, 30 Jul 2011 16:41:58 +0530 Subject: [petsc-users] Will using SBAIJ to solve for all eigenvalues and eigenvectors of a symmetric sparse matrix produced one row at a time be beneficial ? Message-ID: Hi !! I have just started using SLEPc/PETSc. The problem that I am trying to solve is that I have some routines calculating one row at a time of a matrix, which I want to insert in some sparse matrix representation, and then find the next row, insert it, and so on. (This is being done to avoid the need to have one big NxN matrix in memory).The matrix is also symmetric, so the row that the subroutines produce is such that any non-zero element can be present only in that part of the row corresponding to the upper triangle of the matrix. (all others are zero). After I have the matrix in sparse format, I want to find all the eigenvalues and eigenvectors. Till now I have been using the SeqAij matrix and MatSetValues method with preallocation set appropriately, but I am not satisfied with its performance (the way I am doing it). When one row is produced, I insert it into the AIJ matrix, so there are total of N calls to MatSetValues. Also even though the matrix is symmetric, i have to enter the full matrix, as otherwise the eigensolver doesnt seem to produce the right answer (Why ? I set the problem type to EPS_HEP, but still it requires me to store the full matrix ? It is written in documentation that problem type should always be specified so that the eigensolver can take advantage of it (symmetry in this case), but considering that I still have to store the full matrix, it meant 'advantage' only in time, not memory ?....when I set the problem type to EPS_HEP, it means that it will take only the upper or lower triangle right ? why should it require both ? Is this because the matrix type is not symmetric, unlike SBAIJ ?) So, if I use SBAIJ and say, enter some bs rows at a time into the SBAIJ matrix, will it be faster ? Also, in that case would I still have to enter both upper and lower triangle or only one (I hope only one, otherwise the 'S' doesnt make sense to me, but in that case, which one ? The eigensolver doesnt give me an option to specify if the upper or lower triangle is stored in the matrix in case the problem type is EPS_HEP). Will the eigensolver produce the right answers in this case ? (say, storing only upper triangle in SBAIJ, instead of both) Otherwise, while using AIJ, if I insert multiple rows at a time instead of just one (thus reducing the number of calls to MatSetValues, at the cost of more memory), will this be much beneficial ? (Please note that the matrix is not very sparse) Thank you very much in advance !!! Shitij -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat Jul 30 11:26:55 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 30 Jul 2011 16:26:55 +0000 Subject: [petsc-users] Will using SBAIJ to solve for all eigenvalues and eigenvectors of a symmetric sparse matrix produced one row at a time be beneficial ? In-Reply-To: References: Message-ID: On Sat, Jul 30, 2011 at 11:11 AM, Shitij Bhargava wrote: > Hi !! > > I have just started using SLEPc/PETSc. > > The problem that I am trying to solve is that I have some routines > calculating one row at a time of a matrix, which I want to insert in some > sparse matrix representation, and then find the next row, insert it, and so > on. (This is being done to avoid the need to have one big NxN matrix in > memory).The matrix is also symmetric, so the row that the subroutines > produce is such that any non-zero element can be present only in that part > of the row corresponding to the upper triangle of the matrix. (all others > are zero). After I have the matrix in sparse format, I want to find all the > eigenvalues and eigenvectors. > > Till now I have been using the SeqAij matrix and MatSetValues method with > preallocation set appropriately, but I am not satisfied with its performance > (the way I am doing it). When one row is produced, I insert it into the AIJ > matrix, so there are total of N calls to MatSetValues. Also even though the > matrix is symmetric, i have to enter the full matrix, as otherwise the > eigensolver doesnt seem to produce the right answer (Why ? I set the problem > type to EPS_HEP, but still it requires me to store the full matrix ? It is > written in documentation that problem type should always be specified so > that the eigensolver can take advantage of it (symmetry in this case), but > considering that I still have to store the full matrix, it meant 'advantage' > only in time, not memory ?....when I set the problem type to EPS_HEP, it > means that it will take only the upper or lower triangle right ? why should > it require both ? Is this because the matrix type is not symmetric, unlike > SBAIJ ?) > You must set the type to SBAIJ in order to enter only the symmetric part. > So, if I use SBAIJ and say, enter some bs rows at a time into the SBAIJ > matrix, will it be faster ? Also, in that case would I still have to enter > both upper and lower triangle or only one (I hope only one, otherwise the > 'S' doesnt make sense to me, but in that case, which one ? The eigensolver > doesnt give me an option to specify if the upper or lower triangle is stored > in the matrix in case the problem type is EPS_HEP). Will the eigensolver > produce the right answers in this case ? (say, storing only upper triangle > in SBAIJ, instead of both) > I do not think it will be noticeably faster. Value entry is usually not a large part of the time. YOu can check this using -log_summary. http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MATSEQSBAIJ.html#MATSEQSBAIJ show that only the upper triangle is stored. > Otherwise, while using AIJ, if I insert multiple rows at a time instead of > just one (thus reducing the number of calls to MatSetValues, at the cost of > more memory), will this be much beneficial ? (Please note that the matrix is > not very sparse) > I cannot say anything until we see the timing from -log_summary. It is very rare that matrix construction is a factor, if it has been properly allocated, which you can check using -info. Matt > Thank you very much in advance !!! > > > Shitij > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From xyuan at lbl.gov Sat Jul 30 18:44:58 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Sat, 30 Jul 2011 16:44:58 -0700 Subject: [petsc-users] Is there a way to print out the nonzero structure of a matrix? Message-ID: <5401B402-8FA6-4EFF-884A-D3F78E3A129C@lbl.gov> Hello, I would like to print out the nonzero structure of the analytic Jacobian through PETSc, is there any example on that? Thanks very much! Xuefei (Rebecca) Yuan Postdoctoral Fellow Lawrence Berkeley National Laboratory Tel: 1-510-486-7031 From knepley at gmail.com Sat Jul 30 20:01:50 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 31 Jul 2011 01:01:50 +0000 Subject: [petsc-users] Is there a way to print out the nonzero structure of a matrix? In-Reply-To: <5401B402-8FA6-4EFF-884A-D3F78E3A129C@lbl.gov> References: <5401B402-8FA6-4EFF-884A-D3F78E3A129C@lbl.gov> Message-ID: On Sat, Jul 30, 2011 at 11:44 PM, Xuefei (Rebecca) Yuan wrote: > Hello, > > I would like to print out the nonzero structure of the analytic Jacobian > through PETSc, is there any example on that? > -mat_view_draw? Matt > Thanks very much! > > Xuefei (Rebecca) Yuan > Postdoctoral Fellow > Lawrence Berkeley National Laboratory > Tel: 1-510-486-7031 > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Sun Jul 31 02:59:31 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Sun, 31 Jul 2011 00:59:31 -0700 Subject: [petsc-users] Is there a way to print out the nonzero structure of a matrix? In-Reply-To: References: <5401B402-8FA6-4EFF-884A-D3F78E3A129C@lbl.gov> Message-ID: if you dump the matrix in a file, you can use the spy() command of MATLAB. Mohammad On Sat, Jul 30, 2011 at 6:01 PM, Matthew Knepley wrote: > On Sat, Jul 30, 2011 at 11:44 PM, Xuefei (Rebecca) Yuan wrote: > >> Hello, >> >> I would like to print out the nonzero structure of the analytic Jacobian >> through PETSc, is there any example on that? >> > > -mat_view_draw? > > Matt > > >> Thanks very much! >> >> Xuefei (Rebecca) Yuan >> Postdoctoral Fellow >> Lawrence Berkeley National Laboratory >> Tel: 1-510-486-7031 >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Jul 31 11:41:46 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 31 Jul 2011 11:41:46 -0500 Subject: [petsc-users] use Superlu as ilu preconditioner In-Reply-To: References: <4E2F2950.2030701@tuhh.de> Message-ID: On Fri, Jul 29, 2011 at 18:22, Jed Brown wrote: > Are you using the development version of libMesh? I built libMesh with > petsc-dev not long ago, and I send updates when I notice changes in > petsc-dev that affect libMesh. Is it not working now? > I have sent patches to libmesh-devel to support the latest petsc-dev. You can apply them to your svn libmesh checkout or wait for upstream to merge them. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vijay.m at gmail.com Sun Jul 31 23:11:18 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Sun, 31 Jul 2011 23:11:18 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: > The stiff integrators in TS are intended for solving DAE. If you have > consistent initial conditions, then TSGL should just work, perhaps after > turning off error control or specifying your own controller (the built-in > controllers are not robust at present). If you do not have consistent > initial conditions, then they need to be solved for, which we don't > currently have support for, but should be adding reasonably soon. I saw from the documentation that the GL methods were capable of handling DAE systems. But the intention of my question was about the usage of the newton solver with functional constraints (with VI) for a DAE since in a sense, the DAE systems are a composite of an ODE and a constraint. So I was curious whether we could use temporal methods that were not intended for DAEs but for stiff ODEs only and use the VI solver to augment/restrict the solution with constraints. The reason for this is due to the fact that in the creation of a stiff solver, there are additional coefficient constraints that need to be satisfied to be used for a DAE but this is less restrictive for solving ODEs. But looking at the paper that Shri pointed out, this does not seem like an option. I was probably just shooting in the dark there but it just seemed like a neat idea. Vijay On Fri, Jul 29, 2011 at 4:40 PM, Jed Brown wrote: > On Mon, Jul 25, 2011 at 23:40, Vijay S. Mahadevan wrote: >> >> I was also going to ask whether this can be used to solve a DAE by >> decoupling it into a time dependent equation and an algebraic (functional) >> constraint associated with it. > > The stiff integrators in TS are intended for solving DAE. If you have > consistent initial conditions, then TSGL should just work, perhaps after > turning off error control or specifying your own controller (the built-in > controllers are not robust at present). If you do not have consistent > initial conditions, then they need to be solved for, which we don't > currently have support for, but should be adding reasonably soon. From jedbrown at mcs.anl.gov Sun Jul 31 23:14:49 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 31 Jul 2011 22:14:49 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 22:11, Vijay S. Mahadevan wrote: > I saw from the documentation that the GL methods were capable of > handling DAE systems. But the intention of my question was about the > usage of the newton solver with functional constraints (with VI) for a > DAE since in a sense, the DAE systems are a composite of an ODE and a > constraint. So I was curious whether we could use temporal methods > that were not intended for DAEs but for stiff ODEs only and use the VI > solver to augment/restrict the solution with constraints. The reason > for this is due to the fact that in the creation of a stiff solver, > there are additional coefficient constraints that need to be satisfied > to be used for a DAE but this is less restrictive for solving ODEs. > Do you want equality or inequality constraints? Equality constraints are naturally handled by writing a DAE. Inequality constraints would be handled by VI. VI is not currently available through TS, but my understanding is that Shri is looking into this and it will eventually be available there too. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vijay.m at gmail.com Sun Jul 31 23:24:47 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Sun, 31 Jul 2011 23:24:47 -0500 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: > Do you want equality or inequality constraints? Equality constraints are > naturally handled by writing a DAE. I am looking at equality constraints only for now. But I just realized that, the constraints in VI seem only to be on variables but not on a function of it. So I guess the decoupled ODE+constraint might not have worked. On Sun, Jul 31, 2011 at 11:14 PM, Jed Brown wrote: > On Sun, Jul 31, 2011 at 22:11, Vijay S. Mahadevan wrote: >> >> I saw from the documentation that the GL methods were capable of >> handling DAE systems. But the intention of my question was about the >> usage of the newton solver with functional constraints (with VI) for a >> DAE since in a sense, the DAE systems are a composite of an ODE and a >> constraint. So I was curious whether we could use temporal methods >> that were not intended for DAEs but for stiff ODEs only and use the VI >> solver to augment/restrict the solution with constraints. The reason >> for this is due to the fact that in the creation of a stiff solver, >> there are additional coefficient constraints that need to be satisfied >> to be used for a DAE but this is less restrictive for solving ODEs. > > Do you want equality or inequality constraints? Equality constraints are > naturally handled by writing a DAE. Inequality constraints would be handled > by VI. VI is not currently available through TS, but my understanding is > that Shri is looking into this and it will eventually be available there > too. From jedbrown at mcs.anl.gov Sun Jul 31 23:29:51 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 31 Jul 2011 22:29:51 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: References: Message-ID: On Sun, Jul 31, 2011 at 22:24, Vijay S. Mahadevan wrote: > I am looking at equality constraints only for now. > VI stands for Variational Inequality. It wouldn't make sense to use for equality constraints. Just write your system as a DAE. > But I just realized > that, the constraints in VI seem only to be on variables but not on a > function of it. > That just means you have to introduce new variables. Maybe others can comment on this, but I suspect that, as with DAEs, the well-behaved system that you eventually want to solve might come after eliminating those intermediate variables. For DAEs, we just changed the formalism slightly so the user (and any Jacobian information that they might provide) works with a form in which the extra variables have been eliminated. If that is the case, we should look at the API to see if we can hide the intermediate variables from the user (thus letting them set constraints on arbitrary functions of the independent variables). -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Sun Jul 31 23:55:43 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Sun, 31 Jul 2011 23:55:43 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: Message-ID: <779179883.176252.1312174543961.JavaMail.root@zimbra.anl.gov> ----- Original Message ----- On Sun, Jul 31, 2011 at 22:11, Vijay S. Mahadevan < vijay.m at gmail.com > wrote: I saw from the documentation that the GL methods were capable of handling DAE systems. But the intention of my question was about the usage of the newton solver with functional constraints (with VI) for a DAE since in a sense, the DAE systems are a composite of an ODE and a constraint. So I was curious whether we could use temporal methods that were not intended for DAEs but for stiff ODEs only and use the VI solver to augment/restrict the solution with constraints. The reason for this is due to the fact that in the creation of a stiff solver, there are additional coefficient constraints that need to be satisfied to be used for a DAE but this is less restrictive for solving ODEs. Do you want equality or inequality constraints? Equality constraints are naturally handled by writing a DAE. Inequality constraints would be handled by VI. VI is not currently available through TS, but my understanding is that Shri is looking into this and it will eventually be available there too. I have already added TSVISetVariableBounds() by which lower and upper bounds can be set for variables directly through the TS interface. I am currently working on getting an example ready for TS with VI solver. Hopefully, i will add it by this week. Thanks, Shri -------------- next part -------------- An HTML attachment was scrubbed... URL: