From gdiso at ustc.edu Sun May 1 02:40:48 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sun, 1 May 2011 15:40:48 +0800 (CST) Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: <4DBC4DDC.1010604@geology.wisc.edu> References: <4DBC4DDC.1010604@geology.wisc.edu> Message-ID: <22840242.9981304235648381.JavaMail.coremail@mail.ustc.edu> I guess you are using AIJ matrix. The inexact preallocation is a killer to performance. Since the preallcation of AIJ(or CRS) format is organized by row. Any row has an actual column entries exceeded the preallocated number of this row, will cause the whole matrix rebuild! I also suffered much by the this preallocation mechanism. As a result, I am developing a FAIJ matrix. It uses hash table to hold the matrix entry which do not has its preallocated location. And the exact matrix nonzero pattern will be calculated at MatAssemblyEnd, then insert those entries in hash table to their right location. Now the serial version tested ok. But parallel version still buggy. I will commit it to petsc group when I finish it. > > > I having some performance issues with preallocation in a fully > > unstructured FE code. It would be very helpful if those using FE codes > > can comment. > > > > For a problem of size 100K nodes and 600K tet elements (on 1 cpu) > > > > 1. If I calculate the _exact_ number of non-zeros per row (using a > > running list in Fortran) by looping over nodes & elements, the code > > takes 17 mins (to calculate nnz's/per row, assemble and solve). > > 2. If I dont use a running list and simply get the average of the max > > number of nodes a node might be connected to (again by looping over > > nodes & elements but not using a running list) then it takes 8 mins > > 3. If I just magically guess the right value calculated in 2 and use > > that as average nnz per row then it only takes 25 secs. > > > > Basically in all cases Assembly and Solve are very fast (few seconds) > > but the nnz calculation itself (in 2 and 3) takes a long time. How can > > this be cut down? Is there a heuristic way to estimate the number (as > > done in 3) even if it slightly overestimates the nnz's per row or are > > efficient ways to do step 1 or 2. Right now I have do i=1,num_nodes; do > > j=1,num_elements ... which obviously is slow for large number of > > nodes/elements. > > > > Thanks in advance > > Tabrez > > From knepley at gmail.com Sun May 1 09:53:14 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 1 May 2011 09:53:14 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: <4DBC4DDC.1010604@geology.wisc.edu> References: <4DBC4DDC.1010604@geology.wisc.edu> Message-ID: On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali wrote: > Petsc Developers/Users > > I having some performance issues with preallocation in a fully unstructured > FE code. It would be very helpful if those using FE codes can comment. > > For a problem of size 100K nodes and 600K tet elements (on 1 cpu) > > 1. If I calculate the _exact_ number of non-zeros per row (using a running > list in Fortran) by looping over nodes & elements, the code takes 17 mins > (to calculate nnz's/per row, assemble and solve). > 2. If I dont use a running list and simply get the average of the max > number of nodes a node might be connected to (again by looping over nodes & > elements but not using a running list) then it takes 8 mins > 3. If I just magically guess the right value calculated in 2 and use that > as average nnz per row then it only takes 25 secs. > > Basically in all cases Assembly and Solve are very fast (few seconds) but > the nnz calculation itself (in 2 and 3) takes a long time. How can this be > cut down? Is there a heuristic way to estimate the number (as done in 3) > even if it slightly overestimates the nnz's per row or are efficient ways to > do step 1 or 2. Right now I have do i=1,num_nodes; do j=1,num_elements ... > which obviously is slow for large number of nodes/elements. > If you want to see my code doing this, look at include/petscdmmesh.hh:preallocateOperatorNew() which handles the determination of nonzero structure for a FEM operator. It should look mostly like your own code. Matt > Thanks in advance > Tabrez > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From lizs at mail.uc.edu Sun May 1 11:57:15 2011 From: lizs at mail.uc.edu (Li, Zhisong (lizs)) Date: Sun, 1 May 2011 16:57:15 +0000 Subject: [petsc-users] parallel incomplete Cholesky conjugate gradient solver Message-ID: <88D7E3BB7E1960428303E760100374515F1367FF@BL2PRD0103MB055.prod.exchangelabs.com> Hi, Petsc Team, For those elliptic Poisson-style equations, the matrix is often non-symmetrical and costly to solve. And it's often mentioned that the incomplete Cholesky conjugate gradient (ICCG) method will be the best choice for handling these problems. In Petsc, the ICC preconditioner is only implemented in serial processing. And parallel ICC is not available in other packages such as Hypre. Do you know how to achieve an ICCG in parallel processing for an existing Petsc code without too much effort? Thank you very much. Zhisong Li From jed at 59A2.org Sun May 1 12:07:17 2011 From: jed at 59A2.org (Jed Brown) Date: Sun, 1 May 2011 19:07:17 +0200 Subject: [petsc-users] parallel incomplete Cholesky conjugate gradient solver In-Reply-To: <88D7E3BB7E1960428303E760100374515F1367FF@BL2PRD0103MB055.prod.exchangelabs.com> References: <88D7E3BB7E1960428303E760100374515F1367FF@BL2PRD0103MB055.prod.exchangelabs.com> Message-ID: On Sun, May 1, 2011 at 18:57, Li, Zhisong (lizs) wrote: > For those elliptic Poisson-style equations, the matrix is often > non-symmetrical and costly to solve. And it's often mentioned that the > incomplete Cholesky conjugate gradient (ICCG) method will be the best choice > for handling these problems. > Hardly. Multigrid is much better for "Poisson-style equations". > > In Petsc, the ICC preconditioner is only implemented in serial processing. > And parallel ICC is not available in other packages such as Hypre. > You can use ILU. It is normally the same, but uses more memory. Note that incomplete factorization tends not to scale very well, so you may want to consider multilevel or domain decomposition methods. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilyascfd at gmail.com Mon May 2 02:33:09 2011 From: ilyascfd at gmail.com (ilyas ilyas) Date: Mon, 2 May 2011 10:33:09 +0300 Subject: [petsc-users] working with different size of arrays in a single DA In-Reply-To: References: Message-ID: Jed, I was looking at http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/snes/examples/tutorials/ex31.c Although ghost cell support is available in DMDA, DMDA in this example is defined to have PERIODIC BC, in order to implement/force inflow/outflow bc's. Do you have an idea why it is defined so? or Am I missing something? Thanks again, Ilyas. 2011/4/30 Jed Brown > On Sat, Apr 30, 2011 at 13:17, ilyas ilyas wrote: > >> I guess the thing that makes it complicated application of boundary >> conditions. >> Since XYZGHOSTED is not available in fortran, >> I am using XYZPERIODIC in order to implement bc's, as it is done in ex31.c >> in SNES >> Combining XYZPERIODIC with larger block size is still not much clear for >> me. >> >> By the way, what is the current status of DMDACreate3D >> and other DMDA routines and their fortran support ? >> According to ex11f90.F and ex22f.F in the manual page of DMDACreate3d , >> these routine(s) provide ghost cell support for fortran in 3D ? >> If ghost cell support is available, implementation would be relatively >> easy >> > > There is Fortran support for periodic boundaries. Also, petsc-dev has > support for ghost cells even when they do not imply any wrapping, > see DMDA_BOUNDARY_GHOST. > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/DM/DMDACreate3d.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 2 03:59:27 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 2 May 2011 10:59:27 +0200 Subject: [petsc-users] working with different size of arrays in a single DA In-Reply-To: References: Message-ID: On Mon, May 2, 2011 at 09:33, ilyas ilyas wrote: > I was looking at > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/snes/examples/tutorials/ex31.c > > Although ghost cell support is available in DMDA, DMDA in this example is > defined to have PERIODIC BC, > in order to implement/force inflow/outflow bc's. Do you have an idea why it > is defined so? or Am I missing something? > It just wasn't changed since DMDA_BOUNDARY_GHOSTED was added. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Mon May 2 08:16:51 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Mon, 02 May 2011 08:16:51 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: References: <4DBC4DDC.1010604@geology.wisc.edu> Message-ID: <4DBEAEC3.7020708@geology.wisc.edu> Is there a way I can use this and other mesh routines from Fortran? The manual doesn't say much on this. Tabrez On 05/01/2011 09:53 AM, Matthew Knepley wrote: > On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali > wrote: > > Petsc Developers/Users > > I having some performance issues with preallocation in a fully > unstructured FE code. It would be very helpful if those using FE > codes can comment. > > For a problem of size 100K nodes and 600K tet elements (on 1 cpu) > > 1. If I calculate the _exact_ number of non-zeros per row (using a > running list in Fortran) by looping over nodes & elements, the > code takes 17 mins (to calculate nnz's/per row, assemble and solve). > 2. If I dont use a running list and simply get the average of the > max number of nodes a node might be connected to (again by looping > over nodes & elements but not using a running list) then it takes > 8 mins > 3. If I just magically guess the right value calculated in 2 and > use that as average nnz per row then it only takes 25 secs. > > Basically in all cases Assembly and Solve are very fast (few > seconds) but the nnz calculation itself (in 2 and 3) takes a long > time. How can this be cut down? Is there a heuristic way to > estimate the number (as done in 3) even if it slightly > overestimates the nnz's per row or are efficient ways to do step 1 > or 2. Right now I have do i=1,num_nodes; do j=1,num_elements ... > which obviously is slow for large number of nodes/elements. > > > If you want to see my code doing this, look at > > include/petscdmmesh.hh:preallocateOperatorNew() > > which handles the determination of nonzero structure for a FEM > operator. It should look mostly > like your own code. > > Matt > > Thanks in advance > Tabrez > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From austin at txcorp.com Mon May 2 11:03:49 2011 From: austin at txcorp.com (Travis Austin) Date: Mon, 2 May 2011 09:03:49 -0700 Subject: [petsc-users] ML Statistics like BoomerAMG Print Statistics Message-ID: <4A4F2FD5-7131-40DD-AE88-21AAE94F0AAF@txcorp.com> I'm looking for an option to ML that gives information about the hierarchy of matrices constructed by ML like that which is available with BoomerAMG using the -pc_hypre_print_boomeramg_statistics. In HYPRE one can use this to get operator complexity etc that tells you the cost of the AMG algorithm. It seems that there are far fewer options with ML (probably because it has not been a part of PETSc for as long) and I have not yet found an option like -pc_ml_print_statistics. Does anyone know of such an option? Thanks, Travis ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Travis Austin, Ph.D. Tech-X Corporation 5621 Arapahoe Ave, Suite A Boulder, CO 80303 austin at txcorp.com ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 2 11:23:27 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 2 May 2011 18:23:27 +0200 Subject: [petsc-users] ML Statistics like BoomerAMG Print Statistics In-Reply-To: <4A4F2FD5-7131-40DD-AE88-21AAE94F0AAF@txcorp.com> References: <4A4F2FD5-7131-40DD-AE88-21AAE94F0AAF@txcorp.com> Message-ID: On Mon, May 2, 2011 at 18:03, Travis Austin wrote: > > I'm looking for an option to ML that gives information about the hierarchy > of matrices constructed by ML > like that which is available with BoomerAMG using the > -pc_hypre_print_boomeramg_statistics. > > In HYPRE one can use this to get operator complexity etc that tells you the > cost of the AMG algorithm. > This estimate is mostly because Hypre gives you almost no choices about smoothers. You can get some information by increasing -pc_ml_PrintLevel. The hierarchy itself is managed by PETSc, so you can control all aspects of smoothers on levels, etc., -ksp_view will show you the whole hierarchy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.skates82 at gmail.com Tue May 3 12:01:11 2011 From: m.skates82 at gmail.com (Nun ion) Date: Tue, 3 May 2011 12:01:11 -0500 Subject: [petsc-users] Complex spmv performance Message-ID: Hello PETSc developers, I understand PETSc replaces the scalar data type with a complex type for all solvers, and that configuring with fortran kernels improves performance, however it is unclear to me why in .../ksp/ksp/examples/tutorials/ex11.c the dimension of the matrix is set to: dim = n*n. If the matrix entries are stored in CSR can one interpret a matrix with complex entries as a vector where each entry is complex-valued or does this approach create 'two' matrices one with the real value the other with the imaginary part? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Tue May 3 12:06:57 2011 From: rlmackie862 at gmail.com (Randall Mackie) Date: Tue, 3 May 2011 10:06:57 -0700 Subject: [petsc-users] question about petsc and gpus Message-ID: I have access to a computer with a GPU, and I wanted to play around a bit with petsc on this gpu. So I have compiled petsc-dev with cusp, thrust, and cuda to give GPU support, and I am ready to run some examples. However, something is not clear to me: how does petsc know which gpu to run something on? For example, this machine has two gpus: one for the display, and then one compute gpu. So I see /dev/nvidia0 and /dev/nvidia1. If I just issue the command from the petsc web page, will it run it on the right gpu? Also, what if a computer has two compute gpus? Does petsc use both of them or only one? Thanks in advance, Randy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Tue May 3 12:09:55 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 3 May 2011 19:09:55 +0200 Subject: [petsc-users] Complex spmv performance In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 19:01, Nun ion wrote: > I understand PETSc replaces the scalar data type with a complex type for > all solvers, and that configuring with fortran kernels improves performance, > however it is unclear to me why in .../ksp/ksp/examples/tutorials/ex11.c the > dimension of the matrix is set to: dim = n*n. > It is solving a 2D problem, therefore that is the length of the vector. > If the matrix entries are stored in CSR can one interpret a matrix with > complex entries as a vector where each entry is complex-valued or does this > approach create 'two' matrices one with the real value the other with the > imaginary part? > They are stored with fields interlaced, but you shouldn't think about it as a "vector" except for very special things like trying to minimize energy of the rows subject to sparsity constraints. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gaurish108 at gmail.com Tue May 3 17:30:54 2011 From: gaurish108 at gmail.com (Gaurish Telang) Date: Tue, 3 May 2011 18:30:54 -0400 Subject: [petsc-users] PETSc-GPU Message-ID: Hi, is PETSc able to use more than one GPU for doing computational work? So does the current PETSc GPU model support multiple processors and multiple GPU's, 1 processor multiple GPU's or multiple -processors and 1 GPU ? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue May 3 19:48:53 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 3 May 2011 19:48:53 -0500 Subject: [petsc-users] PETSc-GPU In-Reply-To: References: Message-ID: <15FB6949-06B2-4356-823C-F54AD2A59770@mcs.anl.gov> On May 3, 2011, at 5:30 PM, Gaurish Telang wrote: > Hi, is PETSc able to use more than one GPU for doing computational work? > > So does the current PETSc GPU model support multiple processors and multiple GPU's, 1 processor multiple GPU's or multiple -processors and 1 GPU ? We currently support several processors each using a single GPU partner. We do not know how to associated two GPUs attached to the same processor with two processes. You'll need to study Nvidia documentation to see how to do this. Barry > > Thanks. > From bsmith at mcs.anl.gov Tue May 3 20:31:48 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 3 May 2011 20:31:48 -0500 Subject: [petsc-users] question about petsc and gpus In-Reply-To: References: Message-ID: <6059EE76-82A7-4CCB-AB77-A275594662A7@mcs.anl.gov> On May 3, 2011, at 12:06 PM, Randall Mackie wrote: > I have access to a computer with a GPU, and I wanted to play around a bit with petsc on this gpu. > So I have compiled petsc-dev with cusp, thrust, and cuda to give GPU support, and I am ready > to run some examples. > > However, something is not clear to me: how does petsc know which gpu to run something on? > For example, this machine has two gpus: one for the display, and then one compute gpu. > So I see /dev/nvidia0 and /dev/nvidia1. If I just issue the command from the petsc web page, > will it run it on the right gpu? > > > Also, what if a computer has two compute gpus? Does petsc use both of them or only one? > > > Thanks in advance, > > Randy Randy, This is something we don't know anything about. How to assign GPUs to processes etc. You'll need to read the Nvidia documentation to find out how to do this sort of thing. If you figure this out, we'll add it to our documentations. Barry From agrayver at gfz-potsdam.de Wed May 4 05:23:47 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Wed, 04 May 2011 12:23:47 +0200 Subject: [petsc-users] question about petsc and gpus In-Reply-To: References: Message-ID: <4DC12933.1040400@gfz-potsdam.de> Hello Randall, As far as I know, if you don't set particular device the device with index 0 in cuda device array will be used (at least tried to be). Otherwise, you can use cudaSetDevice or cudaChooseDevice Regards, Alexander On 03.05.2011 19:06, Randall Mackie wrote: > I have access to a computer with a GPU, and I wanted to play around a > bit with petsc on this gpu. > So I have compiled petsc-dev with cusp, thrust, and cuda to give GPU > support, and I am ready > to run some examples. > > However, something is not clear to me: how does petsc know which gpu > to run something on? > For example, this machine has two gpus: one for the display, and then > one compute gpu. > So I see /dev/nvidia0 and /dev/nvidia1. If I just issue the command > from the petsc web page, > will it run it on the right gpu? > > > Also, what if a computer has two compute gpus? Does petsc use both of > them or only one? > > > Thanks in advance, > > Randy From vishy at stat.purdue.edu Fri May 6 00:37:50 2011 From: vishy at stat.purdue.edu (S V N Vishwanathan) Date: Fri, 06 May 2011 01:37:50 -0400 Subject: [petsc-users] PetscSortRealWithPermutation causes segfaults Message-ID: <87hb98xuc1.wl%vishy@stat.purdue.edu> Hi I am using PetscSortRealWithPermutation to sort an array with around 200K elements and it causes a segfault. STL sort is able to handle this array without any problems. I looked at the gdb backtrace. It primarily seems to be because the stack is exhausted because of recursive calls to PetscSortRealWithPermutation_Private. I am wondering if it might not be a better idea to implement PetscSortRealWithPermutation_Private without recursion to avoid these errors? vishy From jed at 59A2.org Fri May 6 02:43:36 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 6 May 2011 09:43:36 +0200 Subject: [petsc-users] PetscSortRealWithPermutation causes segfaults In-Reply-To: <87hb98xuc1.wl%vishy@stat.purdue.edu> References: <87hb98xuc1.wl%vishy@stat.purdue.edu> Message-ID: On Fri, May 6, 2011 at 07:37, S V N Vishwanathan wrote: > I am using PetscSortRealWithPermutation to sort an array with around > 200K elements and it causes a segfault. > > STL sort is able to handle this array without any problems. > > I looked at the gdb backtrace. It primarily seems to be because the > stack is exhausted because of recursive calls to > PetscSortRealWithPermutation_Private. I am wondering if it might not be > a better idea to implement PetscSortRealWithPermutation_Private without > recursion to avoid these errors? > Non-recursive may still need O(n) memory because a "stack" needs to be stored, and it just masks the real problem which is decay to quadratic behavior. This issue of hitting worst case of quicksort came up before with PetscSortInt. It would be interesting to know if the pivot selection in PetscSortInt_Private would fix the worst-case behavior that you see? If so, then the same changes could be made to the other sorting routines. Better solutions involve putting real work into finding good pivots or falling back to heap sort ("introsort") when bad behavior is detected. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vishy at stat.purdue.edu Fri May 6 11:59:01 2011 From: vishy at stat.purdue.edu (S V N Vishwanathan) Date: Fri, 06 May 2011 12:59:01 -0400 Subject: [petsc-users] PetscSortRealWithPermutation causes segfaults In-Reply-To: References: <87hb98xuc1.wl%vishy@stat.purdue.edu> Message-ID: <87r58bdaui.wl%vishy@stat.purdue.edu> > This issue of hitting worst case of quicksort came up before with > PetscSortInt. It would be interesting to know if the pivot selection > in PetscSortInt_Private would fix the worst-case behavior that you > see? If so, then the same changes could be made to the other sorting > routines. I did a bit more investigation and found that this worst case is happening when all the elements of the array are the same. I am detecting this in my code and avoid calling the sort as a stop gap measure. I don't think there will be trouble with a O(n) memory in a queue but a recursive call overflows the stack. vishy From jed at 59A2.org Fri May 6 12:38:04 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 6 May 2011 19:38:04 +0200 Subject: [petsc-users] PetscSortRealWithPermutation causes segfaults In-Reply-To: <87r58bdaui.wl%vishy@stat.purdue.edu> References: <87hb98xuc1.wl%vishy@stat.purdue.edu> <87r58bdaui.wl%vishy@stat.purdue.edu> Message-ID: On Fri, May 6, 2011 at 18:59, S V N Vishwanathan wrote: > I did a bit more investigation and found that this worst case is > happening when all the elements of the array are the same. I am > detecting this in my code and avoid calling the sort as a stop gap > measure. > > I don't think there will be trouble with a O(n) memory in a queue but a > recursive call overflows the stack. > The problem is that if you are blowing the stack, then you are seeing quadratic behavior which is not acceptable either. So "putting the stack on the heap" (managing the stack manually) isn't an acceptable solution, the quadratic complexity has to be fixed. I believe that this worst-case behavior is pretty hard to hit with the pivot choice in PetscSortInt_Private, I know it is a problem with the pivot choice in the other variants, they just didn't get updated when PetscSortInt was updated. Always dynamically allocating memory is not ideal because it costs more on some architectures. Tracking the stack depth and switching to manual management of the stack is about the same amount of programming effort as falling back to heap sort, but the latter guarantees log-linear complexity and no dynamic memory allocation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Fri May 6 13:15:17 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Fri, 06 May 2011 20:15:17 +0200 Subject: [petsc-users] Erro in solution Message-ID: <1304705717.4144.26.camel@linux-sclt.site> Dear PETSc users, I have recently started to use PETSc in our code. As the first try, I assemble the matrix row by row and then put each non-zero element in each row into PETSc matrix. Then I start to solve the equation. I use PETSc on a multi-core machine, so I will not use MPI in my program. I compiled PETSc without MPI. The PETSc parts of my code are: Mat Mp; Vec xp, bp; KSP ksp; PetscErrorCode ierr; ierr=MatCreate(PETSC_COMM_WORLD, &Mp); ierr=MatSetSizes(Mp, Msize, Msize, Msize, Msize); ierr=MatSetType(Mp, MATSEQAIJ); ierr=MatSetUp(Mp); ierr=VecCreate(PETSC_COMM_WORLD, &xp); ierr=VecSetSizes(xp, Msize, Msize); ierr=VecSetFromOptions(xp); ierr=VecCreate(PETSC_COMM_WORLD, &bp); ierr=VecSetSizes(bp, Msize, Msize); ierr=VecSetFromOptions(bp); . . . // each non zero element of row "i" and column "k" is put in // PETSc matrix if (M_row[k]!=0.0) ierr=MatSetValues(Mp, 1, &i, 1, &k, &M_row[k], ADD_VALUES); ierr=MatAssemblyBegin(Mp, MAT_FINAL_ASSEMBLY); ierr=MatAssemblyEnd(Mp, MAT_FINAL_ASSEMBLY); . . . // copy right-hand-side to PETSc vector for (int i=0; i References: <1304705717.4144.26.camel@linux-sclt.site> Message-ID: <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> Danesh, Some of the PETSc solvers (like the default ILU) require that you put entries on all the diagonal locations of the matrix, even if they are zero. So you should explicitly put a 0 on the diagonal locations that have 0 and the issue will go away. Barry On May 6, 2011, at 1:15 PM, Danesh Daroui wrote: > > Dear PETSc users, > > I have recently started to use PETSc in our code. As the first try, I > assemble the matrix row by row and then put each non-zero element in > each row into PETSc matrix. Then I start to solve the equation. I use > PETSc on a multi-core machine, so I will not use MPI in my program. I > compiled PETSc without MPI. The PETSc parts of my code are: > > > Mat Mp; > Vec xp, bp; > KSP ksp; > PetscErrorCode ierr; > > ierr=MatCreate(PETSC_COMM_WORLD, &Mp); > ierr=MatSetSizes(Mp, Msize, Msize, Msize, Msize); > ierr=MatSetType(Mp, MATSEQAIJ); > ierr=MatSetUp(Mp); > > ierr=VecCreate(PETSC_COMM_WORLD, &xp); > ierr=VecSetSizes(xp, Msize, Msize); > ierr=VecSetFromOptions(xp); > > ierr=VecCreate(PETSC_COMM_WORLD, &bp); > ierr=VecSetSizes(bp, Msize, Msize); > ierr=VecSetFromOptions(bp); > > . > . > . > > // each non zero element of row "i" and column "k" is put in > // PETSc matrix > if (M_row[k]!=0.0) > ierr=MatSetValues(Mp, 1, &i, 1, &k, &M_row[k], > ADD_VALUES); > > > ierr=MatAssemblyBegin(Mp, MAT_FINAL_ASSEMBLY); > ierr=MatAssemblyEnd(Mp, MAT_FINAL_ASSEMBLY); > > . > . > . > > // copy right-hand-side to PETSc vector > for (int i=0; i VecSetValue(bp, i, b[i], INSERT_VALUES); > > VecAssemblyBegin(bp); > VecAssemblyEnd(bp); > > ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); > ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); > ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, > PETSC_DEFAULT); > ierr=KSPSetFromOptions(ksp); > ierr=KSPSolve(ksp, bp, xp); > > > > > > The problem is that whenever I run the code I get following output: > > > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Object is in wrong state! > [0]PETSC ERROR: Matrix is missing diagonal entry 309! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > 13:37:48 CDT 2011 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Unknown Name on a linux-gnu named linux-sclt.site by dan > Fri May 6 19:34:19 2011 > [0]PETSC ERROR: Libraries linked > from /home/danesh/petsc-3.1-p8/linux-gnu-intel/lib > [0]PETSC ERROR: Configure run at Fri May 6 15:50:26 2011 > [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel > --with-scalar-type=complex --with-clanguage=c++ --with-mpi=0 > --with-cc=icc --with-fc=gfortran --with-cxx=icpc > --with-blas-lapack-dir=/home/danesh/intel/mkl/lib/intel64/ > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() line 1627 in > src/mat/impls/aij/seq/aijfact.c > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1731 in > src/mat/impls/aij/seq/aijfact.c > [0]PETSC ERROR: MatILUFactorSymbolic() line 5464 in > src/mat/interface/matrix.c > [0]PETSC ERROR: PCSetUp_ILU() line 204 in > src/ksp/pc/impls/factor/ilu/ilu.c > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c > > > > I am sure that the matrix is correct and the system is not singular > because the system can be solved using LAPACK's direct solver. I > probably format the PETSc matrix incorrectly or some PETSc parameters > are wrong. I would appreciate if you give me any clue about possible > problem(s). > > Regards, > > Danesh > > From xdliang at gmail.com Fri May 6 15:56:43 2011 From: xdliang at gmail.com (Xiangdong Liang) Date: Fri, 6 May 2011 16:56:43 -0400 Subject: [petsc-users] Get and Set the k-th diagonal elements Message-ID: Hello everyone, I am wondering whether there is some function like MatGetDiagonal to get the k-th diagonal (above or below the main diagonal) of the matrix A. If I have to write my own function, what built-in function should I use to get a particular element of A? It seems that MatGetValues returns a block of matrix. I guess if I only extract one element once, it would not be efficient. Any suggestions? Thanks. Best, Xiangdong From bsmith at mcs.anl.gov Fri May 6 16:04:20 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 6 May 2011 16:04:20 -0500 Subject: [petsc-users] Get and Set the k-th diagonal elements In-Reply-To: References: Message-ID: <52341AF4-700B-4ECC-8FA4-237ACC7ED10A@mcs.anl.gov> How are you planning to use these diagonals? This might help us make suggestions as to what is a reasonable approach. Accessing diagonals is not a natural operation for sparse matrices stored by rows or columns. Barry On May 6, 2011, at 3:56 PM, Xiangdong Liang wrote: > Hello everyone, > > I am wondering whether there is some function like MatGetDiagonal to > get the k-th diagonal (above or below the main diagonal) of the matrix > A. If I have to write my own function, what built-in function should I > use to get a particular element of A? It seems that MatGetValues > returns a block of matrix. I guess if I only extract one element once, > it would not be efficient. Any suggestions? Thanks. > > Best, > Xiangdong From xdliang at gmail.com Fri May 6 16:23:57 2011 From: xdliang at gmail.com (Xiangdong Liang) Date: Fri, 6 May 2011 17:23:57 -0400 Subject: [petsc-users] Get and Set the k-th diagonal elements In-Reply-To: <52341AF4-700B-4ECC-8FA4-237ACC7ED10A@mcs.anl.gov> References: <52341AF4-700B-4ECC-8FA4-237ACC7ED10A@mcs.anl.gov> Message-ID: Thanks for your response, Barry. I am going to solve A_n x = b for different A_n's. The difference between A_n's are only the main diagonal and the k-th diagonal. I want to generate the matrix A once, and each time modify the main and k-th diagonals to get a new A_n. That's the reason I am asking for efficient way to get and set k-th diagonal. I may not need to get the k-th diagonal if I keep the adding difference between each n. However, I still need to set the k-th diagonal anyway. More precisely, suppose my N-by-N matrix A=[a1, a2; a3, a4], where a1, a2, a3 and a4 are n/2-by-n/2 sub-matrix. I am trying to modify the diagonals of a1, a2, a3 and a4, which is the main and n/2-th diagonal of A. Thanks. Xiangdong On Fri, May 6, 2011 at 5:04 PM, Barry Smith wrote: > > ?How are you planning to use these diagonals? This might help us make suggestions as to what is a reasonable approach. Accessing diagonals is not a natural operation for sparse matrices stored by rows or columns. > > ? Barry > > > > On May 6, 2011, at 3:56 PM, Xiangdong Liang wrote: > >> Hello everyone, >> >> I am wondering whether there is some function like MatGetDiagonal to >> get the k-th diagonal (above or below the main diagonal) of the matrix >> A. If I have to write my own function, what built-in function should I >> use to get a particular element of A? It seems that MatGetValues >> returns a block of matrix. I guess if I only extract one element once, >> it would not be efficient. Any suggestions? Thanks. >> >> Best, >> Xiangdong > > From bsmith at mcs.anl.gov Fri May 6 21:08:55 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 6 May 2011 21:08:55 -0500 Subject: [petsc-users] Get and Set the k-th diagonal elements In-Reply-To: References: <52341AF4-700B-4ECC-8FA4-237ACC7ED10A@mcs.anl.gov> Message-ID: Are the diagonals constant values, or does each row have a separate value? Barry On May 6, 2011, at 4:23 PM, Xiangdong Liang wrote: > Thanks for your response, Barry. I am going to solve A_n x = b for > different A_n's. The difference between A_n's are only the main > diagonal and the k-th diagonal. I want to generate the matrix A once, > and each time modify the main and k-th diagonals to get a new A_n. > That's the reason I am asking for efficient way to get and set k-th > diagonal. I may not need to get the k-th diagonal if I keep the adding > difference between each n. However, I still need to set the k-th > diagonal anyway. > > More precisely, suppose my N-by-N matrix A=[a1, a2; a3, a4], where a1, > a2, a3 and a4 are n/2-by-n/2 sub-matrix. I am trying to modify the > diagonals of a1, a2, a3 and a4, which is the main and n/2-th diagonal > of A. > > Thanks. > > Xiangdong > > > > On Fri, May 6, 2011 at 5:04 PM, Barry Smith wrote: >> >> How are you planning to use these diagonals? This might help us make suggestions as to what is a reasonable approach. Accessing diagonals is not a natural operation for sparse matrices stored by rows or columns. >> >> Barry >> >> >> >> On May 6, 2011, at 3:56 PM, Xiangdong Liang wrote: >> >>> Hello everyone, >>> >>> I am wondering whether there is some function like MatGetDiagonal to >>> get the k-th diagonal (above or below the main diagonal) of the matrix >>> A. If I have to write my own function, what built-in function should I >>> use to get a particular element of A? It seems that MatGetValues >>> returns a block of matrix. I guess if I only extract one element once, >>> it would not be efficient. Any suggestions? Thanks. >>> >>> Best, >>> Xiangdong >> >> From amesga1 at tigers.lsu.edu Sat May 7 10:22:59 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Sat, 7 May 2011 10:22:59 -0500 Subject: [petsc-users] checking for hypre in preprocessor Message-ID: <4D38ABF7-C3E4-45AD-8F2F-2D8B79F2FE35@tigers.lsu.edu> Dear all, Is there a way to check if PETSc is compiled with Hypre using C preprocessor directives? (something like #ifdef WITH_HYPRE) Best, Ata Mesgarnejad From milanm at student.ethz.ch Sat May 7 10:46:31 2011 From: milanm at student.ethz.ch (Milan Mitrovic) Date: Sat, 7 May 2011 17:46:31 +0200 Subject: [petsc-users] Using PETSc with a custom data structure Message-ID: Hello! I am working on a project that uses the ppm library and I would like to use petsc because I need the non-linear solvers. As far as I've seen the solvers were made to work with the petsc vector data type, but I need to use it with a different data structure that is already scattered onto different processors. Is there a way to create a custom version of petsc vector that would just wrap around the existing data? Something similar to how sundials provides a way to wrap custom data types. If there is a way to do this cleanly the wrappers could get included into ppm, so I thought that I should ask what the best way of doing this would be. Thanks in advance, Milan Mitrovic From jed at 59A2.org Sat May 7 11:04:18 2011 From: jed at 59A2.org (Jed Brown) Date: Sat, 7 May 2011 18:04:18 +0200 Subject: [petsc-users] checking for hypre in preprocessor In-Reply-To: <4D38ABF7-C3E4-45AD-8F2F-2D8B79F2FE35@tigers.lsu.edu> References: <4D38ABF7-C3E4-45AD-8F2F-2D8B79F2FE35@tigers.lsu.edu> Message-ID: On Sat, May 7, 2011 at 17:22, Ataollah Mesgarnejad wrote: > Is there a way to check if PETSc is compiled with Hypre using C > preprocessor directives? (something like #ifdef WITH_HYPRE) #ifdef PETSC_HAVE_HYPRE -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sat May 7 11:22:35 2011 From: jed at 59A2.org (Jed Brown) Date: Sat, 7 May 2011 18:22:35 +0200 Subject: [petsc-users] Using PETSc with a custom data structure In-Reply-To: References: Message-ID: On Sat, May 7, 2011 at 17:46, Milan Mitrovic wrote: > As far as I've seen the solvers were made to work with the petsc > vector data type, but I need to use it with a different data structure > that is already scattered onto different processors. Is there a way to > create a custom version of petsc vector that would just wrap around > the existing data? Something similar to how sundials provides a way to > wrap custom data types. > > If there is a way to do this cleanly the wrappers could get included > into ppm, so I thought that I should ask what the best way of doing > this would be. > You absolutely can, wrap your existing data, but let's discuss your needs first. How is the existing data stored? If it is in a contiguous array, then you can use VecCreateMPIWithArray or VecCreateGhostWithArray or VecPlaceArray as appropriate to have PETSc use your data structures without a copy. If it is not in a contiguous array, then you can essentially write your own (partial) Vec implementation and PETSc will use that. However, PETSc matrices (Mat) will not magically work with your new Vec implementation unless you provide access to the values as a contiguous array. This will force you to manage your own matrices and will significantly limit the number of preconditioners you have available. Also, many Vec operations will be much faster if you just make a copy so that they can all use a contiguous array. The bottom line is that if your storage format is not contiguous, then it is very likely that end-to-end performance will be better if you make a copy, and it will give you access to many more solver combinations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From milanm at student.ethz.ch Sat May 7 12:26:41 2011 From: milanm at student.ethz.ch (Milan Mitrovic) Date: Sat, 7 May 2011 19:26:41 +0200 Subject: [petsc-users] Using PETSc with a custom data structure In-Reply-To: References: Message-ID: The storage is contiguous but the domain decomposition and processor assignment are controlled by the library. And it is also in fortran 95. I will try to make it work with the WithArray methods and I'll let you know how it goes :) Thanks for the quick reply! From bartlomiej.wach at yahoo.pl Sun May 8 11:54:43 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Sun, 8 May 2011 17:54:43 +0100 (BST) Subject: [petsc-users] Global variables In-Reply-To: Message-ID: <591208.2336.qm@web28307.mail.ukl.yahoo.com> Hello, In my application, process of rank 0 reads an integer from a file, can I broadcast it without using a Vec? Reagards Bart?omiej Wach -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sun May 8 11:59:00 2011 From: jed at 59A2.org (Jed Brown) Date: Sun, 8 May 2011 18:59:00 +0200 Subject: [petsc-users] Global variables In-Reply-To: <591208.2336.qm@web28307.mail.ukl.yahoo.com> References: <591208.2336.qm@web28307.mail.ukl.yahoo.com> Message-ID: On Sun, May 8, 2011 at 18:54, Bart?omiej W wrote: > Hello, > > In my application, process of rank 0 reads an integer from a file, can I > broadcast it without using a Vec? > Sure, int myint; if (!rank) { myint = from_file(...); } MPI_Bcast(&myint,1,MPI_INT,0,PETSC_COMM_WORLD); -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Mon May 9 04:04:24 2011 From: gdiso at ustc.edu (Gong Ding) Date: Mon, 9 May 2011 17:04:24 +0800 (CST) Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version Message-ID: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Hi I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. Is there any function like ?PetscBaseType? exist? FAIJ matrix can hold its type name as well as its base type name. I hope FAIJ can be accepted by petsc 3.2. Thanks. Gong Ding From bsmith at mcs.anl.gov Mon May 9 07:32:04 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 07:32:04 -0500 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Message-ID: <249AF5F6-4AAF-4331-98D7-36BCEB13C18B@mcs.anl.gov> On May 9, 2011, at 4:04 AM, Gong Ding wrote: > Hi > I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. > > The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. > Cool! This is a very nice feature; thanks for writing it. > However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: > ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); > FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. > > Is there any function like ?PetscBaseType? exist? FAIJ matrix can hold its type name as well as its base type name. We don't have a good general mechanism for handling this situation. I'll think about what can be done for this. > > I hope FAIJ can be accepted by petsc 3.2. Please see http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html#Sending_patches_to_update_the_ for how to prepare a patch that we can add to petsc-dev. Barry > Thanks. > > Gong Ding > > > > > From bsmith at mcs.anl.gov Mon May 9 07:35:41 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 07:35:41 -0500 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <249AF5F6-4AAF-4331-98D7-36BCEB13C18B@mcs.anl.gov> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <249AF5F6-4AAF-4331-98D7-36BCEB13C18B@mcs.anl.gov> Message-ID: Also please read Chapter 5 of http://www.mcs.anl.gov/petsc/petsc-as/developers/developers.pdf and try to make the new source code confirm to the PETSc style as much as possible. Thanks Barry On May 9, 2011, at 7:32 AM, Barry Smith wrote: > > On May 9, 2011, at 4:04 AM, Gong Ding wrote: > >> Hi >> I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. >> >> The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. >> > Cool! This is a very nice feature; thanks for writing it. > >> However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: >> ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); >> FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. >> >> Is there any function like ?PetscBaseType? exist? FAIJ matrix can hold its type name as well as its base type name. > > We don't have a good general mechanism for handling this situation. I'll think about what can be done for this. > >> >> I hope FAIJ can be accepted by petsc 3.2. > > Please see http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html#Sending_patches_to_update_the_ for how to prepare a patch that we can add to petsc-dev. > > Barry > >> Thanks. >> >> Gong Ding >> >> >> >> >> > From milan.v.mitrovic at gmail.com Sat May 7 10:44:10 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Sat, 7 May 2011 17:44:10 +0200 Subject: [petsc-users] Using PETSc with custom data structures Message-ID: Hello! I am working on a project that uses the ppm library and I would like to use petsc because I need the non-linear solvers. As far as I've seen the solvers were made to work with the petsc vector data type, but I need to use it with a different data structure that is already scattered onto different processors. Is there a way to create a custom version of petsc vector that would just wrap around the existing data? Something similar to how sundials provides a way to wrap custom data types. If there is a way to do this cleanly the wrappers could get included into ppm, so I thought that I should ask what the best way of doing this would be. Thanks in advance, Milan Mitrovic From dan at ltu.se Sun May 8 07:41:57 2011 From: dan at ltu.se (dan at ltu.se) Date: Sun, 8 May 2011 14:41:57 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> Message-ID: <1304858517.4dc68f95d7805@webmail.ltu.se> Hi, I checked my code and there is no zero on the diagonal. I also modified the code so if there is any zero on the diagonal it will be stored anyway. After this change the problem still exists! I am not sure if I have called PETSc functions correctly or in correct sequence. But anyway, it returns the error: [0]PETSC ERROR: Object is in wrong state! that and the rest of the error message is exactly as I posted before. Any idea to solve this problem? Thanks, D. Quoting Barry Smith : > > Danesh, > > Some of the PETSc solvers (like the default ILU) require that you put > entries on all the diagonal locations of the matrix, even if they are zero. > So you should explicitly put a 0 on the diagonal locations that have 0 and > the issue will go away. > > Barry > > On May 6, 2011, at 1:15 PM, Danesh Daroui wrote: > > > > > Dear PETSc users, > > > > I have recently started to use PETSc in our code. As the first try, I > > assemble the matrix row by row and then put each non-zero element in > > each row into PETSc matrix. Then I start to solve the equation. I use > > PETSc on a multi-core machine, so I will not use MPI in my program. I > > compiled PETSc without MPI. The PETSc parts of my code are: > > > > > > Mat Mp; > > Vec xp, bp; > > KSP ksp; > > PetscErrorCode ierr; > > > > ierr=MatCreate(PETSC_COMM_WORLD, &Mp); > > ierr=MatSetSizes(Mp, Msize, Msize, Msize, Msize); > > ierr=MatSetType(Mp, MATSEQAIJ); > > ierr=MatSetUp(Mp); > > > > ierr=VecCreate(PETSC_COMM_WORLD, &xp); > > ierr=VecSetSizes(xp, Msize, Msize); > > ierr=VecSetFromOptions(xp); > > > > ierr=VecCreate(PETSC_COMM_WORLD, &bp); > > ierr=VecSetSizes(bp, Msize, Msize); > > ierr=VecSetFromOptions(bp); > > > > . > > . > > . > > > > // each non zero element of row "i" and column "k" is put in > > // PETSc matrix > > if (M_row[k]!=0.0) > > ierr=MatSetValues(Mp, 1, &i, 1, &k, &M_row[k], > > ADD_VALUES); > > > > > > ierr=MatAssemblyBegin(Mp, MAT_FINAL_ASSEMBLY); > > ierr=MatAssemblyEnd(Mp, MAT_FINAL_ASSEMBLY); > > > > . > > . > > . > > > > // copy right-hand-side to PETSc vector > > for (int i=0; i > VecSetValue(bp, i, b[i], INSERT_VALUES); > > > > VecAssemblyBegin(bp); > > VecAssemblyEnd(bp); > > > > ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); > > ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); > > ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, > > PETSC_DEFAULT); > > ierr=KSPSetFromOptions(ksp); > > ierr=KSPSolve(ksp, bp, xp); > > > > > > > > > > > > The problem is that whenever I run the code I get following output: > > > > > > > > > > [0]PETSC ERROR: --------------------- Error Message > > ------------------------------------ > > [0]PETSC ERROR: Object is in wrong state! > > [0]PETSC ERROR: Matrix is missing diagonal entry 309! > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > > 13:37:48 CDT 2011 > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > [0]PETSC ERROR: See docs/index.html for manual pages. > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Unknown Name on a linux-gnu named linux-sclt.site by dan > > Fri May 6 19:34:19 2011 > > [0]PETSC ERROR: Libraries linked > > from /home/danesh/petsc-3.1-p8/linux-gnu-intel/lib > > [0]PETSC ERROR: Configure run at Fri May 6 15:50:26 2011 > > [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel > > --with-scalar-type=complex --with-clanguage=c++ --with-mpi=0 > > --with-cc=icc --with-fc=gfortran --with-cxx=icpc > > --with-blas-lapack-dir=/home/danesh/intel/mkl/lib/intel64/ > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() line 1627 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1731 in > > src/mat/impls/aij/seq/aijfact.c > > [0]PETSC ERROR: MatILUFactorSymbolic() line 5464 in > > src/mat/interface/matrix.c > > [0]PETSC ERROR: PCSetUp_ILU() line 204 in > > src/ksp/pc/impls/factor/ilu/ilu.c > > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c > > > > > > > > I am sure that the matrix is correct and the system is not singular > > because the system can be solved using LAPACK's direct solver. I > > probably format the PETSc matrix incorrectly or some PETSc parameters > > are wrong. I would appreciate if you give me any clue about possible > > problem(s). > > > > Regards, > > > > Danesh > > > > > > > From danesh.daroui at ltu.se Mon May 9 11:02:04 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Mon, 09 May 2011 18:02:04 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1304858517.4dc68f95d7805@webmail.ltu.se> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> Message-ID: <1304956924.7025.7.camel@linux-sclt.site> Hi again, The issue is solved and it was due to a bug in my code. But now the problem is that PETSc is extremely slow when I try to solve even a small equation. I fill the matrix and right-hand-side and call the solver as below: ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, PETSC_DEFAULT); ierr=KSPSetFromOptions(ksp); ierr=KSPSolve(ksp, bp, xp); The Msize is the size of right-hand-side vector. I use C++ and am running PETSc in sequential mode so no MPI is used, but anyway, it shouldn't be that slow! Am I right or I am missing something? Thanks, Danesh On Sun, 2011-05-08 at 14:41 +0200, dan at ltu.se wrote: > Hi, > > I checked my code and there is no zero on the diagonal. I also modified the code > so if there is any zero on the diagonal it will be stored anyway. After this > change the problem still exists! I am not sure if I have called PETSc functions > correctly or in correct sequence. But anyway, it returns the error: > > [0]PETSC ERROR: Object is in wrong state! > > > that and the rest of the error message is exactly as I posted before. Any idea > to solve this problem? > > Thanks, > > D. > > > > Quoting Barry Smith : > > > > > Danesh, > > > > Some of the PETSc solvers (like the default ILU) require that you put > > entries on all the diagonal locations of the matrix, even if they are zero. > > So you should explicitly put a 0 on the diagonal locations that have 0 and > > the issue will go away. > > > > Barry > > > > On May 6, 2011, at 1:15 PM, Danesh Daroui wrote: > > > > > > > > Dear PETSc users, > > > > > > I have recently started to use PETSc in our code. As the first try, I > > > assemble the matrix row by row and then put each non-zero element in > > > each row into PETSc matrix. Then I start to solve the equation. I use > > > PETSc on a multi-core machine, so I will not use MPI in my program. I > > > compiled PETSc without MPI. The PETSc parts of my code are: > > > > > > > > > Mat Mp; > > > Vec xp, bp; > > > KSP ksp; > > > PetscErrorCode ierr; > > > > > > ierr=MatCreate(PETSC_COMM_WORLD, &Mp); > > > ierr=MatSetSizes(Mp, Msize, Msize, Msize, Msize); > > > ierr=MatSetType(Mp, MATSEQAIJ); > > > ierr=MatSetUp(Mp); > > > > > > ierr=VecCreate(PETSC_COMM_WORLD, &xp); > > > ierr=VecSetSizes(xp, Msize, Msize); > > > ierr=VecSetFromOptions(xp); > > > > > > ierr=VecCreate(PETSC_COMM_WORLD, &bp); > > > ierr=VecSetSizes(bp, Msize, Msize); > > > ierr=VecSetFromOptions(bp); > > > > > > . > > > . > > > . > > > > > > // each non zero element of row "i" and column "k" is put in > > > // PETSc matrix > > > if (M_row[k]!=0.0) > > > ierr=MatSetValues(Mp, 1, &i, 1, &k, &M_row[k], > > > ADD_VALUES); > > > > > > > > > ierr=MatAssemblyBegin(Mp, MAT_FINAL_ASSEMBLY); > > > ierr=MatAssemblyEnd(Mp, MAT_FINAL_ASSEMBLY); > > > > > > . > > > . > > > . > > > > > > // copy right-hand-side to PETSc vector > > > for (int i=0; i > > VecSetValue(bp, i, b[i], INSERT_VALUES); > > > > > > VecAssemblyBegin(bp); > > > VecAssemblyEnd(bp); > > > > > > ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); > > > ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); > > > ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, > > > PETSC_DEFAULT); > > > ierr=KSPSetFromOptions(ksp); > > > ierr=KSPSolve(ksp, bp, xp); > > > > > > > > > > > > > > > > > > The problem is that whenever I run the code I get following output: > > > > > > > > > > > > > > > [0]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > [0]PETSC ERROR: Object is in wrong state! > > > [0]PETSC ERROR: Matrix is missing diagonal entry 309! > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 > > > 13:37:48 CDT 2011 > > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > [0]PETSC ERROR: See docs/index.html for manual pages. > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: Unknown Name on a linux-gnu named linux-sclt.site by dan > > > Fri May 6 19:34:19 2011 > > > [0]PETSC ERROR: Libraries linked > > > from /home/danesh/petsc-3.1-p8/linux-gnu-intel/lib > > > [0]PETSC ERROR: Configure run at Fri May 6 15:50:26 2011 > > > [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel > > > --with-scalar-type=complex --with-clanguage=c++ --with-mpi=0 > > > --with-cc=icc --with-fc=gfortran --with-cxx=icpc > > > --with-blas-lapack-dir=/home/danesh/intel/mkl/lib/intel64/ > > > [0]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() line 1627 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1731 in > > > src/mat/impls/aij/seq/aijfact.c > > > [0]PETSC ERROR: MatILUFactorSymbolic() line 5464 in > > > src/mat/interface/matrix.c > > > [0]PETSC ERROR: PCSetUp_ILU() line 204 in > > > src/ksp/pc/impls/factor/ilu/ilu.c > > > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > > > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > > > [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c > > > > > > > > > > > > I am sure that the matrix is correct and the system is not singular > > > because the system can be solved using LAPACK's direct solver. I > > > probably format the PETSc matrix incorrectly or some PETSc parameters > > > are wrong. I would appreciate if you give me any clue about possible > > > problem(s). > > > > > > Regards, > > > > > > Danesh > > > > > > > > > > > > From jed at 59A2.org Mon May 9 11:22:21 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 9 May 2011 18:22:21 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1304956924.7025.7.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> Message-ID: On Mon, May 9, 2011 at 18:02, Danesh Daroui wrote: > it > shouldn't be that slow! Am I right or I am missing something? > What equations are you solving? Did you build PETSc --with-debugging=0? Run with -ksp_view -log_summary and send the output. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Mon May 9 11:29:25 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Mon, 09 May 2011 18:29:25 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> Message-ID: <1304958565.7025.11.camel@linux-sclt.site> Hi, No I didn't put "no-debugging" when I compiled. I will re-compile it. I am solving a set of Maxwell equations in integral form. I know that my coefficient matrix has a large condition number too. Would you please explain more about "-ksp_view" and "-log_summary"? How can I use these options? I think the problem is deeper than some sort of compilation flags. Can I get number of iterations and know whether the solution has converged or not? In my case, the slow convergence can be due to badly conditioned matrices. I expected to use default preconditioner which should ILU I guess. Regards, Danesh On Mon, 2011-05-09 at 18:22 +0200, Jed Brown wrote: > On Mon, May 9, 2011 at 18:02, Danesh Daroui > wrote: > it > shouldn't be that slow! Am I right or I am missing something? > > What equations are you solving? Did you build PETSc > --with-debugging=0? Run with -ksp_view -log_summary and send the > output. From bsmith at mcs.anl.gov Mon May 9 11:35:25 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 11:35:25 -0500 Subject: [petsc-users] Erro in solution In-Reply-To: <1304956924.7025.7.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> Message-ID: The code is very slow for one of two reasons 1) the matrix preallocation is not correct and it is spending a great deal of time in the MatSetValues() calls. To check this run the code with -info and grep the output for malloc. My guess is that you are not preallocating for the diagonal and hence the preallocation is not enough. See http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#efficient-assembly 2) the convergence of the solver is very slow. Run with -ksp_monitor and see how many iterations it is taking to converge. Is it thousands? If so, you will need to select a better preconditioner or use a direct solver -pc_type lu. Barry On May 9, 2011, at 11:02 AM, Danesh Daroui wrote: > > Hi again, > > The issue is solved and it was due to a bug in my code. But now the > problem is that PETSc is extremely slow when I try to solve even a small > equation. I fill the matrix and right-hand-side and call the solver as > below: > > ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); > ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); > ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, > PETSC_DEFAULT); > ierr=KSPSetFromOptions(ksp); > ierr=KSPSolve(ksp, bp, xp); > > The Msize is the size of right-hand-side vector. I use C++ and am > running PETSc in sequential mode so no MPI is used, but anyway, it > shouldn't be that slow! Am I right or I am missing something? > > Thanks, > > Danesh > > > > > On Sun, 2011-05-08 at 14:41 +0200, dan at ltu.se wrote: >> Hi, >> >> I checked my code and there is no zero on the diagonal. I also modified the code >> so if there is any zero on the diagonal it will be stored anyway. After this >> change the problem still exists! I am not sure if I have called PETSc functions >> correctly or in correct sequence. But anyway, it returns the error: >> >> [0]PETSC ERROR: Object is in wrong state! >> >> >> that and the rest of the error message is exactly as I posted before. Any idea >> to solve this problem? >> >> Thanks, >> >> D. >> >> >> >> Quoting Barry Smith : >> >>> >>> Danesh, >>> >>> Some of the PETSc solvers (like the default ILU) require that you put >>> entries on all the diagonal locations of the matrix, even if they are zero. >>> So you should explicitly put a 0 on the diagonal locations that have 0 and >>> the issue will go away. >>> >>> Barry >>> >>> On May 6, 2011, at 1:15 PM, Danesh Daroui wrote: >>> >>>> >>>> Dear PETSc users, >>>> >>>> I have recently started to use PETSc in our code. As the first try, I >>>> assemble the matrix row by row and then put each non-zero element in >>>> each row into PETSc matrix. Then I start to solve the equation. I use >>>> PETSc on a multi-core machine, so I will not use MPI in my program. I >>>> compiled PETSc without MPI. The PETSc parts of my code are: >>>> >>>> >>>> Mat Mp; >>>> Vec xp, bp; >>>> KSP ksp; >>>> PetscErrorCode ierr; >>>> >>>> ierr=MatCreate(PETSC_COMM_WORLD, &Mp); >>>> ierr=MatSetSizes(Mp, Msize, Msize, Msize, Msize); >>>> ierr=MatSetType(Mp, MATSEQAIJ); >>>> ierr=MatSetUp(Mp); >>>> >>>> ierr=VecCreate(PETSC_COMM_WORLD, &xp); >>>> ierr=VecSetSizes(xp, Msize, Msize); >>>> ierr=VecSetFromOptions(xp); >>>> >>>> ierr=VecCreate(PETSC_COMM_WORLD, &bp); >>>> ierr=VecSetSizes(bp, Msize, Msize); >>>> ierr=VecSetFromOptions(bp); >>>> >>>> . >>>> . >>>> . >>>> >>>> // each non zero element of row "i" and column "k" is put in >>>> // PETSc matrix >>>> if (M_row[k]!=0.0) >>>> ierr=MatSetValues(Mp, 1, &i, 1, &k, &M_row[k], >>>> ADD_VALUES); >>>> >>>> >>>> ierr=MatAssemblyBegin(Mp, MAT_FINAL_ASSEMBLY); >>>> ierr=MatAssemblyEnd(Mp, MAT_FINAL_ASSEMBLY); >>>> >>>> . >>>> . >>>> . >>>> >>>> // copy right-hand-side to PETSc vector >>>> for (int i=0; i>>> VecSetValue(bp, i, b[i], INSERT_VALUES); >>>> >>>> VecAssemblyBegin(bp); >>>> VecAssemblyEnd(bp); >>>> >>>> ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); >>>> ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); >>>> ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, >>>> PETSC_DEFAULT); >>>> ierr=KSPSetFromOptions(ksp); >>>> ierr=KSPSolve(ksp, bp, xp); >>>> >>>> >>>> >>>> >>>> >>>> The problem is that whenever I run the code I get following output: >>>> >>>> >>>> >>>> >>>> [0]PETSC ERROR: --------------------- Error Message >>>> ------------------------------------ >>>> [0]PETSC ERROR: Object is in wrong state! >>>> [0]PETSC ERROR: Matrix is missing diagonal entry 309! >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 >>>> 13:37:48 CDT 2011 >>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>> [0]PETSC ERROR: See docs/index.html for manual pages. >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: Unknown Name on a linux-gnu named linux-sclt.site by dan >>>> Fri May 6 19:34:19 2011 >>>> [0]PETSC ERROR: Libraries linked >>>> from /home/danesh/petsc-3.1-p8/linux-gnu-intel/lib >>>> [0]PETSC ERROR: Configure run at Fri May 6 15:50:26 2011 >>>> [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel >>>> --with-scalar-type=complex --with-clanguage=c++ --with-mpi=0 >>>> --with-cc=icc --with-fc=gfortran --with-cxx=icpc >>>> --with-blas-lapack-dir=/home/danesh/intel/mkl/lib/intel64/ >>>> [0]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() line 1627 in >>>> src/mat/impls/aij/seq/aijfact.c >>>> [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1731 in >>>> src/mat/impls/aij/seq/aijfact.c >>>> [0]PETSC ERROR: MatILUFactorSymbolic() line 5464 in >>>> src/mat/interface/matrix.c >>>> [0]PETSC ERROR: PCSetUp_ILU() line 204 in >>>> src/ksp/pc/impls/factor/ilu/ilu.c >>>> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >>>> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >>>> [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c >>>> >>>> >>>> >>>> I am sure that the matrix is correct and the system is not singular >>>> because the system can be solved using LAPACK's direct solver. I >>>> probably format the PETSc matrix incorrectly or some PETSc parameters >>>> are wrong. I would appreciate if you give me any clue about possible >>>> problem(s). >>>> >>>> Regards, >>>> >>>> Danesh >>>> >>>> >>> >>> >>> > From jed at 59A2.org Mon May 9 11:35:56 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 9 May 2011 18:35:56 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1304958565.7025.11.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304958565.7025.11.camel@linux-sclt.site> Message-ID: On Mon, May 9, 2011 at 18:29, Danesh Daroui wrote: > No I didn't put "no-debugging" when I compiled. I will re-compile it. I > am solving a set of Maxwell equations in integral form. I know that my > coefficient matrix has a large condition number too. Would you please > explain more about "-ksp_view" and "-log_summary"? > Just run your code with these options. They are put in a database by PetscInitialize() and used by the solver when you called KSPSetFromOptions(). > How can I use these > options? I think the problem is deeper than some sort of compilation > flags. Can I get number of iterations and know whether the solution has > converged or not? > Yes, both in code and on the command line. Run with -help for full options and references to the API. -ksp_monitor -ksp_converged_reason will be a good start. > In my case, the slow convergence can be due to badly > conditioned matrices. I expected to use default preconditioner which > should ILU I guess. > Yeah, -ksp_view will show the configuration for the solver. ILU may not work well for your problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Mon May 9 11:50:50 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Mon, 09 May 2011 18:50:50 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> Message-ID: <1304959850.7025.14.camel@linux-sclt.site> On Mon, 2011-05-09 at 11:35 -0500, Barry Smith wrote: > The code is very slow for one of two reasons > > 1) the matrix preallocation is not correct and it is spending a great deal of time in the MatSetValues() calls. To check this run the code with -info and grep the output for malloc. My guess is that you are not preallocating for the diagonal and hence the preallocation is not enough. See http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#efficient-assembly > I don't think that this is the source of the problem. Filling can be more efficient (maybe?) but as I observed, the most time consuming part is when I call KSPSolve. > 2) the convergence of the solver is very slow. Run with -ksp_monitor and see how many iterations it is taking to converge. Is it thousands? If so, you will need to select a better preconditioner or use a direct solver -pc_type lu. > A direct solver? Never! :) I am escaping from time complexity of direct solvers, otherwise I already have a LAPACK based code that works well. > Barry > > > > On May 9, 2011, at 11:02 AM, Danesh Daroui wrote: > > > > > Hi again, > > > > The issue is solved and it was due to a bug in my code. But now the > > problem is that PETSc is extremely slow when I try to solve even a small > > equation. I fill the matrix and right-hand-side and call the solver as > > below: > > > > ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); > > ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); > > ierr=KSPSetTolerances(ksp, 1.e-2/Msize, 1.e-50, PETSC_DEFAULT, > > PETSC_DEFAULT); > > ierr=KSPSetFromOptions(ksp); > > ierr=KSPSolve(ksp, bp, xp); > > > > The Msize is the size of right-hand-side vector. I use C++ and am > > running PETSc in sequential mode so no MPI is used, but anyway, it > > shouldn't be that slow! Am I right or I am missing something? > > > > Thanks, > > > > Danesh > > > > > > > > > > On Sun, 2011-05-08 at 14:41 +0200, dan at ltu.se wrote: > >> Hi, > >> > >> I checked my code and there is no zero on the diagonal. I also modified the code > >> so if there is any zero on the diagonal it will be stored anyway. After this > >> change the problem still exists! I am not sure if I have called PETSc functions > >> correctly or in correct sequence. But anyway, it returns the error: > >> > >> [0]PETSC ERROR: Object is in wrong state! > >> > >> > >> that and the rest of the error message is exactly as I posted before. Any idea > >> to solve this problem? > >> > >> Thanks, > >> > >> D. > >> > >> > >> > >> Quoting Barry Smith : > >> > >>> > >>> Danesh, > >>> > >>> Some of the PETSc solvers (like the default ILU) require that you put > >>> entries on all the diagonal locations of the matrix, even if they are zero. > >>> So you should explicitly put a 0 on the diagonal locations that have 0 and > >>> the issue will go away. > >>> > >>> Barry > >>> From jed at 59A2.org Mon May 9 12:01:33 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 9 May 2011 19:01:33 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1304959850.7025.14.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> Message-ID: On Mon, May 9, 2011 at 18:50, Danesh Daroui wrote: > A direct solver? Never! :) I am escaping from time complexity of direct > solvers, otherwise I already have a LAPACK based code that works well. > Sparse direct solvers are quite different. Scalability is still not optimal, but it's not like for dense matrices. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Mon May 9 12:06:35 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Mon, 09 May 2011 19:06:35 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> Message-ID: <1304960795.7025.16.camel@linux-sclt.site> Hi, Thanks for the tip, but I already have two different version of my solver with PARDISO and MUMPS. Sparse Direct Solvers gave us a great contribution but I need to move to O(n^2) time complexity, So I really need to employ iterative solvers! :) Regards, D. On Mon, 2011-05-09 at 19:01 +0200, Jed Brown wrote: > On Mon, May 9, 2011 at 18:50, Danesh Daroui > wrote: > A direct solver? Never! :) I am escaping from time complexity > of direct > solvers, otherwise I already have a LAPACK based code that > works well. > > Sparse direct solvers are quite different. Scalability is still not > optimal, but it's not like for dense matrices. From jed at 59A2.org Mon May 9 12:57:57 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 9 May 2011 19:57:57 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1304960795.7025.16.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> <1304960795.7025.16.camel@linux-sclt.site> Message-ID: On Mon, May 9, 2011 at 19:06, Danesh Daroui wrote: > Thanks for the tip, but I already have two different version of my > solver with PARDISO and MUMPS. Sparse Direct Solvers gave us a great > contribution but I need to move to O(n^2) time complexity, So I really > need to employ iterative solvers! :) > I'm confused. Is your problem dense? If so, then it doesn't make sense to use sparse solvers. If it is sparse, then the asymptotics for a direct solver are O(n^{3/2}) flops and O(n log n) space in two dimensions and O(n^2) flops and O(n^{4/3}) spare in three dimensions. You can still use PETSc, but sparse preconditioners won't help you. In particular, ILU is just a really crappy direct solver if you use it on a dense matrix. Are there preconditioners for your problem in the literature? Can it be done with a hierarchical method like FMM? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalcinl at gmail.com Mon May 9 18:07:35 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 10 May 2011 02:07:35 +0300 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Message-ID: 2011/5/9 Gong Ding : > Hi > I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. > Great!! Have you thought about extending regular AIJ matrices to use a hash table on first assembly (perhaps activated with an option) and switch back to regular assembly for the next assemblies? I'm assuming here that calls to MatSetValues() fill the hash table even if values are zero. > The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. ?User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. > How did you wrote your hash table? What are the keys and values? Are you using a hash table per row? Or it is actuallyan (i,j)->a hash table? > However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: > ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); > FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. > That's the reason I think that adding this feature to regular AIJ matrices could be better. > Is there any function like ?PetscBaseType? exist? ?FAIJ matrix can hold its type name as well as its base type name. > > I hope FAIJ can be accepted by petsc 3.2. > Thanks. > I think Satish pushed something related to this a few days ago... -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From renzhengyong at gmail.com Mon May 9 18:38:34 2011 From: renzhengyong at gmail.com (Renzhengyong) Date: Tue, 10 May 2011 01:38:34 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> <1304960795.7025.16.camel@linux-sclt.site> Message-ID: HI? I was working on solving Maxwell equations by surface integral approaches. If you are working on this, try to decrease the condition numbers of your integral operators so that it is reasonable to use GMRES solver in a limited number of iterations, which is a need for applying fast multipole method. Zhengyong Zhengyong Ren, Institute of Geophysics, Department of Geoscience, ETH Zurich, CH8092, Zurich, Switzerland. On 2011-5-9, at 19:57, Jed Brown wrote: > On Mon, May 9, 2011 at 19:06, Danesh Daroui wrote: > Thanks for the tip, but I already have two different version of my > solver with PARDISO and MUMPS. Sparse Direct Solvers gave us a great > contribution but I need to move to O(n^2) time complexity, So I really > need to employ iterative solvers! :) > > I'm confused. Is your problem dense? If so, then it doesn't make sense to use sparse solvers. If it is sparse, then the asymptotics for a direct solver are O(n^{3/2}) flops and O(n log n) space in two dimensions and O(n^2) flops and O(n^{4/3}) spare in three dimensions. > > You can still use PETSc, but sparse preconditioners won't help you. In particular, ILU is just a really crappy direct solver if you use it on a dense matrix. Are there preconditioners for your problem in the literature? Can it be done with a hierarchical method like FMM? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 9 20:18:13 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 10 May 2011 09:18:13 +0800 Subject: [petsc-users] nonzero prescribed boundary condition Message-ID: Dear all, I got this question long ago and searched the prior posting but did not find the solution. The question is about nonzero prescribed boundary condition. My understanding is that MatZeroRows() works only for zero prescribed value, not non-zero value. For the non-zero values, we have to remove the rows associated with the boundary, but this will lead to a zero dignal and accordingly the rows in r.h.s should also be removed. My question is that does MatZeroRows() also works for nonzero prescribed boundary and if so how to do it simply? Rong From bsmith at mcs.anl.gov Mon May 9 20:31:09 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 20:31:09 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: Message-ID: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have modified the calling sequence for MatZeroRows() so that it can automatically adjust the appropriate right hand side values for the zeroed rows to support zero or non-zero prescribed boundary conditions easily. Barry On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: > Dear all, > > I got this question long ago and searched the prior posting but did not find the solution. > The question is about nonzero prescribed boundary condition. > My understanding is that MatZeroRows() works only for zero prescribed value, not non-zero value. > For the non-zero values, we have to remove the rows associated with the boundary, but this > will lead to a zero dignal and accordingly the rows in r.h.s should also be removed. > My question is that does MatZeroRows() also works for nonzero prescribed boundary and if so how to do it simply? > > Rong From bsmith at mcs.anl.gov Mon May 9 20:35:52 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 20:35:52 -0500 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Message-ID: On May 9, 2011, at 6:07 PM, Lisandro Dalcin wrote: > 2011/5/9 Gong Ding : >> Hi >> I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. >> > > Great!! Have you thought about extending regular AIJ matrices to use a > hash table on first assembly (perhaps activated with an option) and > switch back to regular assembly for the next assemblies? I'm assuming > here that calls to MatSetValues() fill the hash table even if values > are zero. > >> The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. >> > > How did you wrote your hash table? What are the keys and values? Are > you using a hash table per row? Or it is actuallyan (i,j)->a hash > table? > >> However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: >> ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); >> FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. >> > > That's the reason I think that adding this feature to regular AIJ > matrices could be better. It can be implemented as a "subclass" of AIJ where just a few of the function pointers are changed to provide the hash support. Yes it doesn't make sense to copy the entire class code and so there may not need to be this problem with FAIJ. > >> Is there any function like ?PetscBaseType? exist? FAIJ matrix can hold its type name as well as its base type name. >> >> I hope FAIJ can be accepted by petsc 3.2. >> Thanks. >> > > I think Satish pushed something related to this a few days ago... That was a different matrix class, extensions to BAIJ for faster operations by streaming. That again can be a small subclass addition to BAIJ. Barry > > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169 From rongtian at ncic.ac.cn Mon May 9 21:15:56 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 10 May 2011 10:15:56 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> Message-ID: <040E28514B5D407BA8F7617141426051@rti458laptop> Dear Barry, Thanks a lot for quick answering. I checked the development documents and found the new version of MatZeroRows() does support the nonzero prescribed boundary conditions. I followed up with more details. I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES solvers. I used a displacement-controlled load (as this type of loading works well for all cases). This is the reason the nonzero prescribed boundary came up. In FormJacobian, I modified Jacobian and residual to satisfy the nonzero prescribed boundary. In FormFunction, I modified the solution to the known solution(this should not be necessary as the modified Jacobian and rhs should give the prescribed solution also) Now I found another issue, no matter if I prescried the solution or not in FormFunction, SNES solver always call FormFunction and never call FormJacobian. Of course the solver finally diverged or converged to a zero solution. So my quick follow up question is How a displacement-controled load is done corrently in Petsc 2.3.3? Rong ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Tuesday, May 10, 2011 9:31 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have modified the calling sequence for MatZeroRows() so that it can automatically adjust the appropriate right hand side values for the zeroed rows to support zero or non-zero prescribed boundary conditions easily. Barry On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: > Dear all, > > I got this question long ago and searched the prior posting but did not > find the solution. > The question is about nonzero prescribed boundary condition. > My understanding is that MatZeroRows() works only for zero prescribed > value, not non-zero value. > For the non-zero values, we have to remove the rows associated with the > boundary, but this > will lead to a zero dignal and accordingly the rows in r.h.s should also > be removed. > My question is that does MatZeroRows() also works for nonzero prescribed > boundary and if so how to do it simply? > > Rong From rongtian at ncic.ac.cn Mon May 9 21:19:45 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 10 May 2011 10:19:45 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <040E28514B5D407BA8F7617141426051@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> Message-ID: I forgot to add that when I switched back to force loading, everything works well and the SNES solver converges. ----- Original Message ----- From: "Tian(ICT)" To: "PETSc users list" Sent: Tuesday, May 10, 2011 10:15 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition > Dear Barry, Thanks a lot for quick answering. > I checked the development documents and found the new version of > MatZeroRows() does support the nonzero prescribed boundary conditions. > > I followed up with more details. > I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES > solvers. > I used a displacement-controlled load (as this type of loading works well > for all cases). > This is the reason the nonzero prescribed boundary came up. > > In FormJacobian, I modified Jacobian and residual to satisfy the nonzero > prescribed boundary. > In FormFunction, I modified the solution to the known solution(this should > not be necessary as the modified Jacobian and rhs should give the > prescribed solution also) > > Now I found another issue, no matter if I prescried the solution or not in > FormFunction, > SNES solver always call FormFunction and never call FormJacobian. > Of course the solver finally diverged or converged to a zero solution. > > So my quick follow up question is How a displacement-controled load is > done corrently in Petsc 2.3.3? > > Rong > > ----- Original Message ----- > From: "Barry Smith" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 9:31 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > we have modified the calling sequence for MatZeroRows() so that it can > automatically adjust the appropriate right hand side values for the zeroed > rows to support zero or non-zero prescribed boundary conditions easily. > > Barry > > On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: > >> Dear all, >> >> I got this question long ago and searched the prior posting but did not >> find the solution. >> The question is about nonzero prescribed boundary condition. >> My understanding is that MatZeroRows() works only for zero prescribed >> value, not non-zero value. >> For the non-zero values, we have to remove the rows associated with the >> boundary, but this >> will lead to a zero dignal and accordingly the rows in r.h.s should also >> be removed. >> My question is that does MatZeroRows() also works for nonzero prescribed >> boundary and if so how to do it simply? >> >> Rong > > > From bsmith at mcs.anl.gov Mon May 9 21:22:12 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 9 May 2011 21:22:12 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <040E28514B5D407BA8F7617141426051@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> Message-ID: On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: > Dear Barry, Thanks a lot for quick answering. > I checked the development documents and found the new version of MatZeroRows() does support the nonzero prescribed boundary conditions. > > I followed up with more details. > I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES solvers. > I used a displacement-controlled load (as this type of loading works well for all cases). > This is the reason the nonzero prescribed boundary came up. > > In FormJacobian, I modified Jacobian and residual to satisfy the nonzero prescribed boundary. > In FormFunction, I modified the solution to the known solution(this should not be necessary as the modified Jacobian and rhs should give the prescribed solution also) You should not do it this way. See below. > > Now I found another issue, no matter if I prescried the solution or not in FormFunction, > SNES solver always call FormFunction and never call FormJacobian. The only reason it would not call FormJacobian is if decided that the residual norm was small enough before any Newton steps; for example if the FormFunction() computed exactly the zero function initially. When you run with -snes_monitor -ksp_monitor what does it print for residual norms. > Of course the solver finally diverged or converged to a zero solution. > > So my quick follow up question is How a displacement-controled load is done corrently in Petsc 2.3.3? To do it in 2.3.3 simply have for those components of F() the formula F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for those rows We strongly urge you to upgrade to the latest PETSc before doing anything further. Barry > > Rong > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 9:31 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have modified the calling sequence for MatZeroRows() so that it can automatically adjust the appropriate right hand side values for the zeroed rows to support zero or non-zero prescribed boundary conditions easily. > > Barry > > On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: > >> Dear all, >> >> I got this question long ago and searched the prior posting but did not find the solution. >> The question is about nonzero prescribed boundary condition. >> My understanding is that MatZeroRows() works only for zero prescribed value, not non-zero value. >> For the non-zero values, we have to remove the rows associated with the boundary, but this >> will lead to a zero dignal and accordingly the rows in r.h.s should also be removed. >> My question is that does MatZeroRows() also works for nonzero prescribed boundary and if so how to do it simply? >> >> Rong > > From rongtian at ncic.ac.cn Mon May 9 22:37:51 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 10 May 2011 11:37:51 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop> Message-ID: First, thanks again, the issue was gone. I just followed up with some test results. I have tested SNES using one finite element for a geometric large deformation problem. Those are just the very early test results so they may be not telling what happened exactly. For the displacement controlled load, I found that convergence is much slower than that of force loading. Even worse, linear search is so sensitive to the displacement increment and diverged no matter what the increment size was used (too small incremnt also led to diverged soloution (-6 reason), trust region works well in the sense of not sensitive to the displacement increment, but during each load step, it requires around ten to several tens of Newton interations whereas for the force loading case and the almost same amount of deformation, this is normally 3. This is against my expectation. Any hint? Rong ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Tuesday, May 10, 2011 10:22 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: > Dear Barry, Thanks a lot for quick answering. > I checked the development documents and found the new version of > MatZeroRows() does support the nonzero prescribed boundary conditions. > > I followed up with more details. > I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES > solvers. > I used a displacement-controlled load (as this type of loading works well > for all cases). > This is the reason the nonzero prescribed boundary came up. > > In FormJacobian, I modified Jacobian and residual to satisfy the nonzero > prescribed boundary. > In FormFunction, I modified the solution to the known solution(this should > not be necessary as the modified Jacobian and rhs should give the > prescribed solution also) You should not do it this way. See below. > > Now I found another issue, no matter if I prescried the solution or not in > FormFunction, > SNES solver always call FormFunction and never call FormJacobian. The only reason it would not call FormJacobian is if decided that the residual norm was small enough before any Newton steps; for example if the FormFunction() computed exactly the zero function initially. When you run with -snes_monitor -ksp_monitor what does it print for residual norms. > Of course the solver finally diverged or converged to a zero solution. > > So my quick follow up question is How a displacement-controled load is > done corrently in Petsc 2.3.3? To do it in 2.3.3 simply have for those components of F() the formula F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for those rows We strongly urge you to upgrade to the latest PETSc before doing anything further. Barry > > Rong > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 9:31 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > we have modified the calling sequence for MatZeroRows() so that it can > automatically adjust the appropriate right hand side values for the zeroed > rows to support zero or non-zero prescribed boundary conditions easily. > > Barry > > On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: > >> Dear all, >> >> I got this question long ago and searched the prior posting but did not >> find the solution. >> The question is about nonzero prescribed boundary condition. >> My understanding is that MatZeroRows() works only for zero prescribed >> value, not non-zero value. >> For the non-zero values, we have to remove the rows associated with the >> boundary, but this >> will lead to a zero dignal and accordingly the rows in r.h.s should also >> be removed. >> My question is that does MatZeroRows() also works for nonzero prescribed >> boundary and if so how to do it simply? >> >> Rong > > From rongtian at ncic.ac.cn Mon May 9 22:43:01 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 10 May 2011 11:43:01 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop> Message-ID: <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> by the way, the increment size is like that for a 100 lengh model, the increment is set to 0.05, the engineering strain is around 5%% per load step. This is already too small increment size for a large deformation analysis. a 0.5 increment size leads to both linear search and trust region failed. linear search failed for 0.05 while trust region converges with 17 Newton iterations each load step. Rong ----- Original Message ----- From: "Tian(ICT)" To: "PETSc users list" Sent: Tuesday, May 10, 2011 11:37 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition > First, thanks again, the issue was gone. > > I just followed up with some test results. > I have tested SNES using one finite element for a geometric large > deformation problem. > Those are just the very early test results so they may be not telling what > happened exactly. > For the displacement controlled load, I found that convergence is much > slower than that of force loading. > Even worse, linear search is so sensitive to the displacement increment > and diverged no matter what the increment size was used (too small > incremnt also led to diverged soloution (-6 reason), trust region works > well in the sense of not sensitive to the displacement increment, but > during each load step, it requires around ten to several tens of Newton > interations whereas for the force loading case and the almost same amount > of deformation, this is normally 3. This is against my expectation. Any > hint? > > Rong > > ----- Original Message ----- > From: "Barry Smith" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 10:22 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: > >> Dear Barry, Thanks a lot for quick answering. >> I checked the development documents and found the new version of >> MatZeroRows() does support the nonzero prescribed boundary conditions. >> >> I followed up with more details. >> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >> solvers. >> I used a displacement-controlled load (as this type of loading works well >> for all cases). >> This is the reason the nonzero prescribed boundary came up. >> >> In FormJacobian, I modified Jacobian and residual to satisfy the nonzero >> prescribed boundary. >> In FormFunction, I modified the solution to the known solution(this >> should not be necessary as the modified Jacobian and rhs should give the >> prescribed solution also) > > You should not do it this way. See below. >> >> Now I found another issue, no matter if I prescried the solution or not >> in FormFunction, >> SNES solver always call FormFunction and never call FormJacobian. > > The only reason it would not call FormJacobian is if decided that the > residual norm was small enough before any Newton steps; for example if the > FormFunction() computed exactly the zero function initially. When you run > with -snes_monitor -ksp_monitor what does it print for residual norms. > >> Of course the solver finally diverged or converged to a zero solution. >> >> So my quick follow up question is How a displacement-controled load is >> done corrently in Petsc 2.3.3? > > To do it in 2.3.3 simply have for those components of F() the formula > F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for > those rows > > We strongly urge you to upgrade to the latest PETSc before doing > anything further. > > > Barry > >> >> Rong >> >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Tuesday, May 10, 2011 9:31 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >> we have modified the calling sequence for MatZeroRows() so that it can >> automatically adjust the appropriate right hand side values for the >> zeroed rows to support zero or non-zero prescribed boundary conditions >> easily. >> >> Barry >> >> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >> >>> Dear all, >>> >>> I got this question long ago and searched the prior posting but did not >>> find the solution. >>> The question is about nonzero prescribed boundary condition. >>> My understanding is that MatZeroRows() works only for zero prescribed >>> value, not non-zero value. >>> For the non-zero values, we have to remove the rows associated with the >>> boundary, but this >>> will lead to a zero dignal and accordingly the rows in r.h.s should also >>> be removed. >>> My question is that does MatZeroRows() also works for nonzero prescribed >>> boundary and if so how to do it simply? >>> >>> Rong >> >> > > > From gdiso at ustc.edu Tue May 10 03:16:18 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 10 May 2011 16:16:18 +0800 (CST) Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Message-ID: <5944104.15191305015378866.JavaMail.coremail@mail.ustc.edu> > 2011/5/9 Gong Ding : > > > Hi > > > I created a flexible AIJ matrix type ?named as FAIJ. It has an extra hash table for nonzero which do not have pre-allocated position. And the buffered values in hash table will be flushed to AIJ array at MatAssemblyEnd. Both sequential and parallel version have been implemented and tested. > > > > > > > Great!! Have you thought about extending regular AIJ matrices to use a > > hash table on first assembly (perhaps activated with an option) and > > switch back to regular assembly for the next assemblies? I'm assuming > > here that calls to MatSetValues() fill the hash table even if values > > are zero. MatSetValues will first search position of matrix entry in AIJ array. If entry already exists or having empty space for new entry, only AIJ array will be used. Hash table only be used when no more space exist and MatSeqXAIJReallocateAIJ is required. Add zero is still controled by flag ignorezeroentries. > > > The test results show that for middle size problem ? matrix size around 1M, FAIJ without any pre-allocation still pretty fast. The shortcoming is memory overkill exists. Fortunately, modern computers have larger and larger memory. ?User with FAIJ matrix only needs to provide approximate nonzero pattern or even skip this step. > > > > > > > How did you wrote your hash table? What are the keys and values? Are > > you using a hash table per row? Or it is actuallyan (i,j)->a hash > > table? I just use uthash http://uthash.sourceforge.net. It is a bsd licensed hash table implementation. At present, the hash key is pair. Using row index as hash key, and store each row as dynamic array of col, value is possible. In general, hash table with key is fast enough. > > However, direct solvers such as mumps and superlu_dist will crash. This is caused by type compare functions, i.e.: > > > ierr = PetscTypeCompare((PetscObject)A,MATMPIAIJ,&isAIJ);CHKERRQ(ierr); > > > FAIJ matrix is derived from AIJ, but it has its own type name, i.e. MATMPIFAIJ. The above function will assign isAIJ false. > > > > That's the reason I think that adding this feature to regular AIJ > > matrices could be better. Add hash table to original AIJ avoid the type name problem. However, there are some other matrix types also derived from AIJ. i.e. CSRPERM. As a result, I guess CSRPERM will also crash... I think each sub type (of AIJ matrix, i.e.) should hold its base type. And mumps interface only needs to check the base type. > > Is there any function like ?PetscBaseType? exist? ?FAIJ matrix can hold its type name as well as its base type name. > > > > > > I hope FAIJ can be accepted by petsc 3.2. > > > Thanks. > > > > > > > I think Satish pushed something related to this a few days ago... > > > > > > -- > > Lisandro Dalcin > > --------------- > > CIMEC (INTEC/CONICET-UNL) > > Predio CONICET-Santa Fe > > Colectora RN 168 Km 472, Paraje El Pozo > > 3000 Santa Fe, Argentina > > Tel: +54-342-4511594 (ext 1011) > > Tel/Fax: +54-342-4511169 > > From jed at 59A2.org Tue May 10 04:39:04 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 10 May 2011 11:39:04 +0200 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <5944104.15191305015378866.JavaMail.coremail@mail.ustc.edu> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <5944104.15191305015378866.JavaMail.coremail@mail.ustc.edu> Message-ID: On Tue, May 10, 2011 at 10:16, Gong Ding wrote: > I just use uthash http://uthash.sourceforge.net. It is a bsd licensed hash > table implementation. > I used uthash at one point and later switched to a slightly customized version of khash ( https://github.com/attractivechaos/klib/blob/master/khash.h) which I like much better. See these blog posts by the author: http://attractivechaos.wordpress.com/2008/08/28/comparison-of-hash-table-libraries/ http://attractivechaos.wordpress.com/2008/09/02/implementing-generic-hash-library-in-c/ If PETSc is going to adopt a generic C hash implementation for use in multiple places, it is worth considering. > At present, the hash key is pair. Using row index as hash key, > and store each row as dynamic array of col, value > is possible. > In general, hash table with key is fast enough. > This also has less overhead for when most rows are preallocated correctly. Without benchmarks for specific use cases, I don't see a convincing argument either way. > > Add hash table to original AIJ avoid the type name problem. However, there > are some other matrix types also derived from AIJ. i.e. > CSRPERM. > AIJPERM uses AIJ for assembly so it would also benefit if preallocation spills were handled with the hash. There are also genuinely different formats (e.g. BAIJ) that could benefit from the hash. As perhaps a future matter, it may be worth hooking in at the level of MatSeqXAIJReallocateAIJ. This might not actually be hard, it looks like the right information is already available. I think that hideous macro would need to be turned into a function before doing anything else. As a result, I guess CSRPERM will also crash... > I think each sub type (of AIJ matrix, i.e.) should hold its base type. And > mumps interface only needs to check the base type. > Yes, better handling of base types is needed for several things now so it will be done soon. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Tue May 10 04:43:16 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Tue, 10 May 2011 11:43:16 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> <1304960795.7025.16.camel@linux-sclt.site> Message-ID: <1305020596.3394.6.camel@linux-sclt.site> Yes, My problem is originally dense. That's why I used LAPACK to solve the equation. After some sparsifications, when the problem becomes sparse, I use sparse solvers. As I wrote, I have successfully used direct sparse solvers using other packages and now I am moving to iterative solvers using PETSc. I expected ILU preconditioner to be inappropriate for my needs and that explains slow convergence. I printed out number of iterations and also normalized residual and in slowest case the number of iterations was ~470 and the residual was 6.04524e-5. I experienced faster convergence but with higher residual 0.0005. The solution is also quite inaccurate in most cases and I guess the residual is still high. That's what I guess and the remedy can be using a better preconditioner. As I said, my problem is badly conditioned and this can be observed by plotting eigenvalues spectrum which is not clustered at all and therefore results to a high condition number. Thanks, D. On Mon, 2011-05-09 at 19:57 +0200, Jed Brown wrote: > On Mon, May 9, 2011 at 19:06, Danesh Daroui > wrote: > Thanks for the tip, but I already have two different version > of my > solver with PARDISO and MUMPS. Sparse Direct Solvers gave us a > great > contribution but I need to move to O(n^2) time complexity, So > I really > need to employ iterative solvers! :) > > I'm confused. Is your problem dense? If so, then it doesn't make sense > to use sparse solvers. If it is sparse, then the asymptotics for a > direct solver are O(n^{3/2}) flops and O(n log n) space in two > dimensions and O(n^2) flops and O(n^{4/3}) spare in three dimensions. > > > You can still use PETSc, but sparse preconditioners won't help you. In > particular, ILU is just a really crappy direct solver if you use it on > a dense matrix. Are there preconditioners for your problem in the > literature? Can it be done with a hierarchical method like FMM? From danesh.daroui at ltu.se Tue May 10 04:48:29 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Tue, 10 May 2011 11:48:29 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> <1304960795.7025.16.camel@linux-sclt.site> Message-ID: <1305020909.3394.11.camel@linux-sclt.site> Hi, Yes this is more or less what I am using, however in addition to surface I use volume integrals too for current distributions. Well, I know that condition number should be decreased in general to improve the convergence, but can you please let me know what preconditioner did you use? I have not used Fast Multipole Method because of high approximations specially in lower frequencies. Regards, D. On Tue, 2011-05-10 at 01:38 +0200, Renzhengyong wrote: > HI? > I was working on solving Maxwell equations by surface integral > approaches. If you are working on this, try to decrease the condition > numbers of your integral operators so that it is reasonable to use > GMRES solver in a limited number of iterations, which is a need for > applying fast multipole method. > > > > > Zhengyong > > Zhengyong Ren, > Institute of Geophysics, > Department of Geoscience, > ETH Zurich, CH8092, > Zurich, Switzerland. > > On 2011-5-9, at 19:57, Jed Brown wrote: > > > > > On Mon, May 9, 2011 at 19:06, Danesh Daroui > > wrote: > > Thanks for the tip, but I already have two different version > > of my > > solver with PARDISO and MUMPS. Sparse Direct Solvers gave us > > a great > > contribution but I need to move to O(n^2) time complexity, > > So I really > > need to employ iterative solvers! :) > > > > I'm confused. Is your problem dense? If so, then it doesn't make > > sense to use sparse solvers. If it is sparse, then the asymptotics > > for a direct solver are O(n^{3/2}) flops and O(n log n) space in two > > dimensions and O(n^2) flops and O(n^{4/3}) spare in three > > dimensions. > > > > > > You can still use PETSc, but sparse preconditioners won't help you. > > In particular, ILU is just a really crappy direct solver if you use > > it on a dense matrix. Are there preconditioners for your problem in > > the literature? Can it be done with a hierarchical method like FMM? > > From renzhengyong at gmail.com Tue May 10 06:18:38 2011 From: renzhengyong at gmail.com (RenZhengYong) Date: Tue, 10 May 2011 13:18:38 +0200 Subject: [petsc-users] Erro in solution In-Reply-To: <1305020909.3394.11.camel@linux-sclt.site> References: <1304705717.4144.26.camel@linux-sclt.site> <0C585582-DFD5-48CF-8244-307315FCA90B@mcs.anl.gov> <1304858517.4dc68f95d7805@webmail.ltu.se> <1304956924.7025.7.camel@linux-sclt.site> <1304959850.7025.14.camel@linux-sclt.site> <1304960795.7025.16.camel@linux-sclt.site> <1305020909.3394.11.camel@linux-sclt.site> Message-ID: Hi, I do not use preconditioners for the surface integral formula, because it it well-conditioned. I tested several preconditoners, the convergence rates by using precoditioners are more or less the same as the one without preconditioners. Zhengyong On Tue, May 10, 2011 at 11:48 AM, Danesh Daroui wrote: > > Hi, > > Yes this is more or less what I am using, however in addition to surface > I use volume integrals too for current distributions. Well, I know that > condition number should be decreased in general to improve the > convergence, but can you please let me know what preconditioner did you > use? I have not used Fast Multipole Method because of high > approximations specially in lower frequencies. > > Regards, > > D. > > > On Tue, 2011-05-10 at 01:38 +0200, Renzhengyong wrote: > > HI? > > I was working on solving Maxwell equations by surface integral > > approaches. If you are working on this, try to decrease the condition > > numbers of your integral operators so that it is reasonable to use > > GMRES solver in a limited number of iterations, which is a need for > > applying fast multipole method. > > > > > > > > > > Zhengyong > > > > Zhengyong Ren, > > Institute of Geophysics, > > Department of Geoscience, > > ETH Zurich, CH8092, > > Zurich, Switzerland. > > > > On 2011-5-9, at 19:57, Jed Brown wrote: > > > > > > > > > On Mon, May 9, 2011 at 19:06, Danesh Daroui > > > wrote: > > > Thanks for the tip, but I already have two different version > > > of my > > > solver with PARDISO and MUMPS. Sparse Direct Solvers gave us > > > a great > > > contribution but I need to move to O(n^2) time complexity, > > > So I really > > > need to employ iterative solvers! :) > > > > > > I'm confused. Is your problem dense? If so, then it doesn't make > > > sense to use sparse solvers. If it is sparse, then the asymptotics > > > for a direct solver are O(n^{3/2}) flops and O(n log n) space in two > > > dimensions and O(n^2) flops and O(n^{4/3}) spare in three > > > dimensions. > > > > > > > > > You can still use PETSc, but sparse preconditioners won't help you. > > > In particular, ILU is just a really crappy direct solver if you use > > > it on a dense matrix. Are there preconditioners for your problem in > > > the literature? Can it be done with a hierarchical method like FMM? > > > > > > > -- Zhengyong Ren AUG Group, Institute of Geophysics Department of Geosciences, ETH Zurich NO H 47 Sonneggstrasse 5 CH-8092, Z?rich, Switzerland Tel: +41 44 633 37561 e-mail: zhengyong.ren at aug.ig.erdw.ethz.ch Gmail: renzhengyong at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Tue May 10 06:44:37 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Tue, 10 May 2011 13:44:37 +0200 Subject: [petsc-users] SPAI preconditioner in PETSc Message-ID: <1305027877.3394.19.camel@linux-sclt.site> Hi, I have tried to use SPAI (Sparse Approximate Inverse) preconditioner in PETSc as describes in manual. I choose the preconditioner as below: ierr=KSPCreate(PETSC_COMM_WORLD, &ksp); ierr=KSPSetOperators(ksp, Mp, Mp, DIFFERENT_NONZERO_PATTERN); // conjugate gradient method is used ierr=KSPSetType(ksp, KSPGMRES); // SPAI preconditioner is used ierr=KSPGetPC(ksp, &prec); ierr=PCSetType(prec, PCSPAI); // set up the solver according to the options ierr=KSPSetFromOptions(ksp); ierr=KSPSetUp(ksp); // solve the equation using an iterative solver ierr=KSPSolve(ksp, bp, xp); But when I run my code I get following error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type seehttp://www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html#external! [0]PETSC ERROR: Unable to find requested PC type spai! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Unknown Name on a linux-gnu named linux-sclt.site by dan Tue May 10 13:34:25 2011 [0]PETSC ERROR: Libraries linked from /home/danesh/petsc-3.1-p8/linux-gnu-intel/lib [0]PETSC ERROR: Configure run at Mon May 9 19:30:35 2011 [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel --with-scalar-type=complex --with-clanguage=c++ --with-mpi=0 --with-cc=icc --with-fc=gfortran --with-cxx=icpc --with-blas-lapack-dir=/home/danesh/intel/mkl/lib/intel64/ --with-debugging=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCSetType() line 67 in src/ksp/pc/interface/pcset.c I also checked the SPAI web page and they have also mentioned that there is an interface for PETSc but nothing more. Have anyone used this preconditioner before? Does anybody know how it can be used in PETSc? I know that SPAI uses least squares for approximate inverse and thus it needs a sparsity pattern, so it can simply be called as I did and it needs more parameters which I don't know how can be passed to PETSc. I have also another questions. In my code, even when the solution is converged withing few iterations, the result is far different from correct solution that I have. I am suspecting that there something wrong with matrix filling in my code. Is there anyway to use direct solver in PETSc so I will verify that my matrices are correct before trying iterative solvers? Thanks, Danesh From jed at 59A2.org Tue May 10 06:58:01 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 10 May 2011 13:58:01 +0200 Subject: [petsc-users] SPAI preconditioner in PETSc In-Reply-To: <1305027877.3394.19.camel@linux-sclt.site> References: <1305027877.3394.19.camel@linux-sclt.site> Message-ID: On Tue, May 10, 2011 at 13:44, Danesh Daroui wrote: > I also checked the SPAI web page and they have also mentioned that there > is an interface for PETSc but nothing more. > Configure with --download-spai. You might also try parasails from Hypre (another SPAI implementation), configure with --download-hypre, then run with -pc_type hypre -pc_hypre_type parasails. > > I have also another questions. In my code, even when the solution is > converged withing few iterations, the result is far different from > correct solution that I have. I am suspecting that there something wrong > with matrix filling in my code. Is there anyway to use direct solver in > PETSc so I will verify that my matrices are correct before trying > iterative solvers? > -pc_type lu -------------- next part -------------- An HTML attachment was scrubbed... URL: From hung.thanh.nguyen at uit.no Tue May 10 06:57:32 2011 From: hung.thanh.nguyen at uit.no (Nguyen Hung Thanh) Date: Tue, 10 May 2011 11:57:32 +0000 Subject: [petsc-users] Error of petsc builde or the conflic between Petsc and intel-window Microsoft Visual Stud compiler Message-ID: <9393A2DE2F16CD4ABD3B95167C7DFC1202A288@mbs02.uit.no> Dear Petsc developers/users, It took a while for me to install Petsc, configure and used this library on Intel ?Microsoft Visual platform. And, unfortunately I have not reached the goal yet. I am sorry for that I have again ask you some questions about Petsc configure/ running: 1) I used the configure-options is Hung at TRD-K-ST-008-PC ~/petsc-3.1-p7 $ ./config/configure.py --with-cc='win32fe icl' --with-cxx='win32fe icl' --with -clanguage=cxx --with-fc=0 --with-mpi-include=/home/Hung/IntelMPI/MPI/4.0.1.0 07/ia32/include --with-mpi-lib=/home/Hung/IntelMPI/MPI/4.0.1.007/ia32/lib/imp i.lib --with-blas-lapack-dir=/home/Hung/IntelMKL/ia32/lib NB: impi The configure is seem ok. 2) And, check the Petcs libraries working :$ make PETSC_DIR=/home/Hung/petsc-3.1-p7 PETSC_ARCH=cygwin-cxx-debug test Hung at TRD-K-ST-008-PC ~/petsc-3.1-p7 $ make PETSC_DIR=/home/Hung/petsc-3.1-p7 PETSC_ARCH=cygwin-cxx-debug test Running test examples to verify correct installation Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html [0]PETSC ERROR: --------------------- Error Message ---------------------------- -------- [0]PETSC ERROR: Arguments are incompatible! [0]PETSC ERROR: Memory regions overlap: either use PetscMemmov() or make sure your copy regions and lengths are correct. Length (bytes) 64 first address 0 second address 0! [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 10 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: C:\cygwin\home\Hung\petsc-3.1-p7\src\snes\examples\tutorials\ex1 9.exe on a cygwin-cx named TRD-K-ST-008-PC by Hung Tue May 10 13:35:26 2011 [0]PETSC ERROR: Libraries linked from /home/Hung/petsc-3.1-p7/cygwin-cxx-debug/l ib [0]PETSC ERROR: Configure run at Tue May 10 11:28:56 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe icl" --with-cxx="win32fe ic l" --with-clanguage=cxx --with-fc=0 --with-mpi-include=/home/Hung/IntelMPI/MPI/4 .0.1.007/ia32/include --with-mpi-lib=/home/Hung/IntelMPI/MPI/4.0.1.007/ia32/lib/ impi.lib --with-blas-lapack-dir=/home/Hung/IntelMKL/ia32/lib --useThreads=0 [0]PETSC ERROR: ---------------------------------------------------------------- -------- [0]PETSC ERROR: User provided function() line 1755 in src/vec/is/impls/block/C:\ cygwin\home\Hung\PETSC-~1.1-P\include\petscsys.h [0]PETSC ERROR: ISCreateBlock() line 296 in src/vec/is/impls/block/C:\cygwin\hom e\Hung\PETSC-~1.1-P\src\vec\is\impls\block\block.c [0]PETSC ERROR: DACreate_2D() line 1463 in src/dm/da/src/C:\cygwin\home\Hung\PET SC-~1.1-P\src\dm\da\src\da2.c [0]PETSC ERROR: DASetType() line 48 in src/dm/da/src/C:\cygwin\home\Hung\PETSC-~ 1.1-P\src\dm\da\src\dareg.c [0]PETSC ERROR: DASetTypeFromOptions_Private() line 65 in src/dm/da/src/C:\cygwi n\home\Hung\PETSC-~1.1-P\src\dm\da\src\dacreate.c [0]PETSC ERROR: DASetFromOptions() line 131 in src/dm/da/src/C:\cygwin\home\Hung \PETSC-~1.1-P\src\dm\da\src\dacreate.c [0]PETSC ERROR: DACreate2d() line 1847 in src/dm/da/src/C:\cygwin\home\Hung\PETS C-~1.1-P\src\dm\da\src\da2.c [0]PETSC ERROR: main() line 107 in src/snes/examples/tutorials/C:\cygwin\home\Hu ng\PETSC-~1.1-P\src\snes\examples\TUTORI~1\ex19.c application called MPI_Abort(MPI_COMM_WORLD, 75) - process 0 job aborted: rank: node: exit code[: error message] 0: TRD-K-ST-008-PC.petrell.local: 75: process 0 exited without calling finalize Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI process es..... See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html Completed test examples Please, tell me how I can solve the problem, Regards Hung T Nguyen Numerical analys Physis Derpart. UiT?, Troms? Norway -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Tue May 10 07:21:28 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 10 May 2011 14:21:28 +0200 Subject: [petsc-users] SPAI preconditioner in PETSc In-Reply-To: <1305029896.3394.20.camel@linux-sclt.site> References: <1305027877.3394.19.camel@linux-sclt.site> <1305029896.3394.20.camel@linux-sclt.site> Message-ID: On Tue, May 10, 2011 at 14:18, Danesh Daroui wrote: > Thanks Jed. Can I have both implementations of SPAI and use each of them > whenever I want or I just can have one at time? > Absolutely. You can install PETSc with every external package you might possibly ever want to use, then choose which one you use on any particular run (or per-object if your code solves multiple systems) at run-time with options like -pc_type. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Tue May 10 07:30:51 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Tue, 10 May 2011 14:30:51 +0200 Subject: [petsc-users] SPAI preconditioner in PETSc In-Reply-To: References: <1305027877.3394.19.camel@linux-sclt.site> <1305029896.3394.20.camel@linux-sclt.site> Message-ID: <1305030651.3394.22.camel@linux-sclt.site> I just tried to configure PETSc with SPAI as you said but it says that it is not possible to use SPAI with complex numbers. I use complex numbers in my equations, so is there any other alternatives or solution for this? Thanks, D. On Tue, 2011-05-10 at 14:21 +0200, Jed Brown wrote: > On Tue, May 10, 2011 at 14:18, Danesh Daroui > wrote: > Thanks Jed. Can I have both implementations of SPAI and use > each of them > whenever I want or I just can have one at time? > > Absolutely. You can install PETSc with every external package you > might possibly ever want to use, then choose which one you use on any > particular run (or per-object if your code solves multiple systems) at > run-time with options like -pc_type. From jed at 59A2.org Tue May 10 07:35:05 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 10 May 2011 14:35:05 +0200 Subject: [petsc-users] SPAI preconditioner in PETSc In-Reply-To: <1305030651.3394.22.camel@linux-sclt.site> References: <1305027877.3394.19.camel@linux-sclt.site> <1305029896.3394.20.camel@linux-sclt.site> <1305030651.3394.22.camel@linux-sclt.site> Message-ID: On Tue, May 10, 2011 at 14:30, Danesh Daroui wrote: > I just tried to configure PETSc with SPAI as you said but it says that > it is not possible to use SPAI with complex numbers. I use complex > numbers in my equations, so is there any other alternatives or solution > for this? > SPAI and ParaSails (Hypre) do not support complex. I do not know of a sparse approximate inverse implementation that does. (Unless you have seen a paper saying that it works well for a problem close to yours, I wouldn't worry too much. I have yet to run into a problem for which sparse approximate inverse was the best preconditioner.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue May 10 13:54:03 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 May 2011 13:54:03 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> Message-ID: <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and send the outputs Barry On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: > by the way, the increment size is like that > for a 100 lengh model, the increment is set to 0.05, > the engineering strain is around 5%% per load step. > This is already too small increment size for a large deformation analysis. > a 0.5 increment size leads to both linear search and trust region failed. > linear search failed for 0.05 while trust region converges with 17 Newton iterations each load step. > Rong > > ----- Original Message ----- From: "Tian(ICT)" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 11:37 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > >> First, thanks again, the issue was gone. >> >> I just followed up with some test results. >> I have tested SNES using one finite element for a geometric large deformation problem. >> Those are just the very early test results so they may be not telling what happened exactly. >> For the displacement controlled load, I found that convergence is much slower than that of force loading. >> Even worse, linear search is so sensitive to the displacement increment and diverged no matter what the increment size was used (too small incremnt also led to diverged soloution (-6 reason), trust region works well in the sense of not sensitive to the displacement increment, but during each load step, it requires around ten to several tens of Newton interations whereas for the force loading case and the almost same amount of deformation, this is normally 3. This is against my expectation. Any hint? >> >> Rong >> >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Tuesday, May 10, 2011 10:22 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >> >>> Dear Barry, Thanks a lot for quick answering. >>> I checked the development documents and found the new version of MatZeroRows() does support the nonzero prescribed boundary conditions. >>> >>> I followed up with more details. >>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES solvers. >>> I used a displacement-controlled load (as this type of loading works well for all cases). >>> This is the reason the nonzero prescribed boundary came up. >>> >>> In FormJacobian, I modified Jacobian and residual to satisfy the nonzero prescribed boundary. >>> In FormFunction, I modified the solution to the known solution(this should not be necessary as the modified Jacobian and rhs should give the prescribed solution also) >> >> You should not do it this way. See below. >>> >>> Now I found another issue, no matter if I prescried the solution or not in FormFunction, >>> SNES solver always call FormFunction and never call FormJacobian. >> >> The only reason it would not call FormJacobian is if decided that the residual norm was small enough before any Newton steps; for example if the FormFunction() computed exactly the zero function initially. When you run with -snes_monitor -ksp_monitor what does it print for residual norms. >> >>> Of course the solver finally diverged or converged to a zero solution. >>> >>> So my quick follow up question is How a displacement-controled load is done corrently in Petsc 2.3.3? >> >> To do it in 2.3.3 simply have for those components of F() the formula F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for those rows >> >> We strongly urge you to upgrade to the latest PETSc before doing anything further. >> >> >> Barry >> >>> >>> Rong >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 9:31 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have modified the calling sequence for MatZeroRows() so that it can automatically adjust the appropriate right hand side values for the zeroed rows to support zero or non-zero prescribed boundary conditions easily. >>> >>> Barry >>> >>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>> >>>> Dear all, >>>> >>>> I got this question long ago and searched the prior posting but did not find the solution. >>>> The question is about nonzero prescribed boundary condition. >>>> My understanding is that MatZeroRows() works only for zero prescribed value, not non-zero value. >>>> For the non-zero values, we have to remove the rows associated with the boundary, but this >>>> will lead to a zero dignal and accordingly the rows in r.h.s should also be removed. >>>> My question is that does MatZeroRows() also works for nonzero prescribed boundary and if so how to do it simply? >>>> >>>> Rong >>> >>> >> >> > From Nan.Jia at Dartmouth.edu Tue May 10 14:11:14 2011 From: Nan.Jia at Dartmouth.edu (Nan.Jia at Dartmouth.edu) Date: Tue, 10 May 2011 15:11:14 -0400 Subject: [petsc-users] A beginner question about a PETSc example Message-ID: <20110510151114.n1vpsmqtz4wg0ck0@webmail.dartmouth.edu> Hello all the PETSc users and developers, I just started learning and exploring PETSc recently. I have a problem for the example: .../src/ksp/ksp/examples/ex2f.f I run several times with different numbers of processors. Here are the results. nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ mpiexec -np 1 ./ex2f -m 100 -n 100 Norm of error 0.8936E-02 iterations 65 nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ mpiexec -np 2 ./ex2f -m 100 -n 100 Norm of error 0.3148E-01 iterations 89 nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ mpiexec -np 3 ./ex2f -m 100 -n 100 Norm of error 0.3193E-01 iterations 92 What is weird to me is that with more processors, the norm does not decrease but increases, do does the number of iterations. I also try to compare .../src/ksp/ksp/examples/ex2f.f with .../src/ksp/ksp/examples/ex1f.f to see the difference between the sequential and parallel code in order to understand how PETSc parallelizes the code. This is my first use the PETSc mailing list. Thanks everyone. Best, Nan From hzhang at mcs.anl.gov Tue May 10 14:39:34 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 10 May 2011 14:39:34 -0500 Subject: [petsc-users] A beginner question about a PETSc example In-Reply-To: <20110510151114.n1vpsmqtz4wg0ck0@webmail.dartmouth.edu> References: <20110510151114.n1vpsmqtz4wg0ck0@webmail.dartmouth.edu> Message-ID: See http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#differentiterations Hong On Tue, May 10, 2011 at 2:11 PM, wrote: > Hello all the PETSc users and developers, > I just started learning and exploring PETSc recently. I have a problem for > the example: .../src/ksp/ksp/examples/ex2f.f > I run several times with different numbers of processors. Here are the > results. > > nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ > mpiexec -np 1 ./ex2f -m 100 -n 100 > Norm of error 0.8936E-02 iterations ? ?65 > nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ > mpiexec -np 2 ./ex2f -m 100 -n 100 > Norm of error 0.3148E-01 iterations ? ?89 > nan_jia at mir:/thayerfs/research/anatoly/NAN/Petsc/petsc-3.1-p8/src/ksp/ksp/examples/tutorials$ > mpiexec -np 3 ./ex2f -m 100 -n 100 > Norm of error 0.3193E-01 iterations ? ?92 > > > What is weird to me is that with more processors, the norm does not decrease > but increases, do does the number of iterations. > > I also try to compare .../src/ksp/ksp/examples/ex2f.f with > .../src/ksp/ksp/examples/ex1f.f ?to see the difference between the > sequential and parallel code in order to understand how PETSc parallelizes > the code. > > This is my first use the PETSc mailing list. Thanks everyone. > > Best, > Nan > From amryazanov at gmail.com Tue May 10 16:34:25 2011 From: amryazanov at gmail.com (=?KOI8-R?B?4czFy9PFyiDy0drBzs/X?=) Date: Wed, 11 May 2011 01:34:25 +0400 Subject: [petsc-users] Maximum number of local vectors? Message-ID: Hi, thank you again for all your previous responds. Now I have another question: how many vectors is it possible to get with DAGetLocalVector simultaneously? Is there any restrictions or adjustments? Thank you! Alexey Ryazanov ______________________________________ Nuclear Safety Institute of Russian Academy of Sciences -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue May 10 16:45:18 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 May 2011 16:45:18 -0500 Subject: [petsc-users] Maximum number of local vectors? In-Reply-To: References: Message-ID: <0C2D94ED-05FF-4989-8DB1-2A12E7002D99@mcs.anl.gov> On May 10, 2011, at 4:34 PM, ??????? ??????? wrote: > Hi, thank you again for all your previous responds. > > Now I have another question: how many vectors is it possible to get with DAGetLocalVector simultaneously? Is there any restrictions or adjustments? There is no limit. If you get more than 100 outstanding at a particular time it no longer caches those over 100 outstanding and instead the DARestoreLocalVector() immediately frees the space. But it is completely possible to have hundreds in use at the same time (though it seems strange to me). The Get/Restore paradigm is intended for subroutines that may for a short time need some work vectors. Vectors that you keep around for much of the live of the program normally would be obtained with the Create/Destroy paradigm. The only reason for the Get/Restore code is that users need to work about managing work vectors themselves. Barry > > Thank you! > > Alexey Ryazanov > ______________________________________ > Nuclear Safety Institute of Russian Academy of Sciences > > From amryazanov at gmail.com Tue May 10 17:36:20 2011 From: amryazanov at gmail.com (=?KOI8-R?B?4czFy9PFyiDy0drBzs/X?=) Date: Wed, 11 May 2011 02:36:20 +0400 Subject: [petsc-users] Maximum number of local vectors? In-Reply-To: <0C2D94ED-05FF-4989-8DB1-2A12E7002D99@mcs.anl.gov> References: <0C2D94ED-05FF-4989-8DB1-2A12E7002D99@mcs.anl.gov> Message-ID: Got it! Thank you! Alexey Ryazanov ______________________________________ Nuclear Safety Institute of Russian Academy of Sciences 2011/5/11 Barry Smith > > On May 10, 2011, at 4:34 PM, ??????? ??????? wrote: > > > Hi, thank you again for all your previous responds. > > > > Now I have another question: how many vectors is it possible to get with > DAGetLocalVector simultaneously? Is there any restrictions or adjustments? > > There is no limit. If you get more than 100 outstanding at a particular > time it no longer caches those over 100 outstanding and instead the > DARestoreLocalVector() immediately frees the space. But it is completely > possible to have hundreds in use at the same time (though it seems strange > to me). > > The Get/Restore paradigm is intended for subroutines that may for a short > time need some work vectors. Vectors that you keep around for much of the > live of the program normally would be obtained with the Create/Destroy > paradigm. The only reason for the Get/Restore code is that users need to > work about managing work vectors themselves. > > > Barry > > > > > Thank you! > > > > Alexey Ryazanov > > ______________________________________ > > Nuclear Safety Institute of Russian Academy of Sciences > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Tue May 10 20:14:55 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Wed, 11 May 2011 09:14:55 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> Message-ID: <0B5B1110339742BC8990AB61531039A8@rti458laptop> Dear Barry, here is the output using -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor the attached is the same and for clear reference. Thanks again for helps. atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067811865e-002 0 KSP Residual norm 9.965778978387e-002 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 6.823187455811e-017 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 1 SNES Function norm 6.401926523423e-002 0 KSP Residual norm 8.969200212486e-002 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.106757475780e-016 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 2 SNES Function norm 5.849992149767e-002 0 KSP Residual norm 8.072279488157e-002 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.268750073799e-017 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 3 SNES Function norm 5.376618503592e-002 0 KSP Residual norm 7.265050969883e-002 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.677655733356e-017 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 4 SNES Function norm 4.956894354459e-002 0 KSP Residual norm 6.538545411661e-002 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 9.557004153175e-017 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 5 SNES Function norm 4.575418613137e-002 0 KSP Residual norm 5.884690496914e-002 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.470969262115e-017 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 6 SNES Function norm 4.223022245585e-002 0 KSP Residual norm 5.296221144636e-002 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.255198782390e-017 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 7 SNES Function norm 3.894430065910e-002 0 KSP Residual norm 4.766598785088e-002 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.322615478395e-017 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 8 SNES Function norm 3.586683371135e-002 0 KSP Residual norm 4.289938708067e-002 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 6.181358328498e-017 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 9 SNES Function norm 3.298130202025e-002 0 KSP Residual norm 3.860944676473e-002 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.635174776374e-017 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 10 SNES Function norm 3.027806208930e-002 0 KSP Residual norm 3.474850078591e-002 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.389914053685e-017 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 11 SNES Function norm 2.749422924729e-002 0 KSP Residual norm 3.081350823297e-002 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.640567497647e-017 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 12 SNES Function norm 2.437488247885e-002 0 KSP Residual norm 2.633007441879e-002 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.772331460094e-017 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 13 SNES Function norm 2.079664278637e-002 0 KSP Residual norm 2.104738289397e-002 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.650632708670e-017 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 14 SNES Function norm 1.657344626858e-002 0 KSP Residual norm 1.454141853505e-002 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.129401160070e-017 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 15 SNES Function norm 1.484243752612e-002 0 KSP Residual norm 5.241948491751e-009 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.729506849025e-024 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 16 SNES Function norm 2.828002157497e-008 0 KSP Residual norm 6.042518362322e-015 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 6.272441346127e-030 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 17 SNES Function norm 2.960967020289e-008 STEP 0 (Newton iterations: 17) diverged reason: -6 ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Wednesday, May 11, 2011 2:54 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and send the outputs Barry On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: > by the way, the increment size is like that > for a 100 lengh model, the increment is set to 0.05, > the engineering strain is around 5%% per load step. > This is already too small increment size for a large deformation analysis. > a 0.5 increment size leads to both linear search and trust region failed. > linear search failed for 0.05 while trust region converges with 17 Newton > iterations each load step. > Rong > > ----- Original Message ----- From: "Tian(ICT)" > To: "PETSc users list" > Sent: Tuesday, May 10, 2011 11:37 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > >> First, thanks again, the issue was gone. >> >> I just followed up with some test results. >> I have tested SNES using one finite element for a geometric large >> deformation problem. >> Those are just the very early test results so they may be not telling >> what happened exactly. >> For the displacement controlled load, I found that convergence is much >> slower than that of force loading. >> Even worse, linear search is so sensitive to the displacement increment >> and diverged no matter what the increment size was used (too small >> incremnt also led to diverged soloution (-6 reason), trust region works >> well in the sense of not sensitive to the displacement increment, but >> during each load step, it requires around ten to several tens of Newton >> interations whereas for the force loading case and the almost same amount >> of deformation, this is normally 3. This is against my expectation. Any >> hint? >> >> Rong >> >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Tuesday, May 10, 2011 10:22 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >> >>> Dear Barry, Thanks a lot for quick answering. >>> I checked the development documents and found the new version of >>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>> >>> I followed up with more details. >>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>> solvers. >>> I used a displacement-controlled load (as this type of loading works >>> well for all cases). >>> This is the reason the nonzero prescribed boundary came up. >>> >>> In FormJacobian, I modified Jacobian and residual to satisfy the nonzero >>> prescribed boundary. >>> In FormFunction, I modified the solution to the known solution(this >>> should not be necessary as the modified Jacobian and rhs should give the >>> prescribed solution also) >> >> You should not do it this way. See below. >>> >>> Now I found another issue, no matter if I prescried the solution or not >>> in FormFunction, >>> SNES solver always call FormFunction and never call FormJacobian. >> >> The only reason it would not call FormJacobian is if decided that the >> residual norm was small enough before any Newton steps; for example if >> the FormFunction() computed exactly the zero function initially. When you >> run with -snes_monitor -ksp_monitor what does it print for residual >> norms. >> >>> Of course the solver finally diverged or converged to a zero solution. >>> >>> So my quick follow up question is How a displacement-controled load is >>> done corrently in Petsc 2.3.3? >> >> To do it in 2.3.3 simply have for those components of F() the formula >> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for >> those rows >> >> We strongly urge you to upgrade to the latest PETSc before doing >> anything further. >> >> >> Barry >> >>> >>> Rong >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 9:31 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >>> we have modified the calling sequence for MatZeroRows() so that it can >>> automatically adjust the appropriate right hand side values for the >>> zeroed rows to support zero or non-zero prescribed boundary conditions >>> easily. >>> >>> Barry >>> >>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>> >>>> Dear all, >>>> >>>> I got this question long ago and searched the prior posting but did not >>>> find the solution. >>>> The question is about nonzero prescribed boundary condition. >>>> My understanding is that MatZeroRows() works only for zero prescribed >>>> value, not non-zero value. >>>> For the non-zero values, we have to remove the rows associated with the >>>> boundary, but this >>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>> also be removed. >>>> My question is that does MatZeroRows() also works for nonzero >>>> prescribed boundary and if so how to do it simply? >>>> >>>> Rong >>> >>> >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: aa Type: application/octet-stream Size: 6582 bytes Desc: not available URL: From sylbar.vainbot at gmail.com Tue May 10 21:20:49 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Tue, 10 May 2011 19:20:49 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: Dear Jed, Thank you for your previous comments. I am still trying to design a multigrid preconditionner for the Navier's equation of elasticity. I have defined shell matrices for the interpolation, restriction, smoothing, residual calculation and the fine-level solver (see attached my_green.f90 file with stub functions). I thought that all these shell matrices would be enough to do the job, but upon running KSPSetup(), I get the following error: [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: This matrix type, shell, does not support getting diagonal block! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 14:42:02 CDT 2010 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c regarding your previous comments: "For Jacobi, you can have your MatShell implement MatGetDiagonal and it will work. (...) If you want to use a multiplicative relaxation like SOR, you would have to implement it yourself.", I actually want to use SOR with my smoother shell matrix. What am I doing wrong to make petsc still believe I want to use Jacobi? another related question: with a direct solver, one needs to assign a shell matrix with a KSP object (KSPSetOperators); is it still required when using multigrid? if I don't do so, I get the following error: [0]PETSC ERROR: Null argument, when expecting valid pointer! [0]PETSC ERROR: Null Object: Parameter # 1! [0]PETSC ERROR: MatGetVecs() line 7265 in src/mat/interface/matrix.c [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSetUp() line 199 in src/ksp/ksp/interface/itfunc.c however, I don't understand why I would need this matrix in a multigrid framework. Is there any example out there of using PCMG with shell matrices? Thanks in advance for your response. Best wishes, Sylvain Barbot 2010/10/7 Jed Brown : > On Thu, Oct 7, 2010 at 11:45, Sylvain Barbot > wrote: >> >> questions: >> 1) is it at all possible to specify this mode of operation, from the >> finest to the coarser level, and back? Any examples out there? > > -pc_mg_type multiplicative, or > ??PCMGSetType(pc,PC_MG_MULTIPLICATIVE); > >> >> 2) is it readily possible to use matrix shells with DMMG? I imagine >> the Jacobian matrix may simply be provided as a matrix shell. Is there >> any examples of multi-grid methods with shell matrices online? > > You can do this, but you will have to define a custom smoother. ?For Jacobi, > you can have your MatShell implement MatGetDiagonal and it will work. ?I > thought you could implement MatGetDiagonalBlock for PBJacobi, but it's not > currently written that way (though it should be and that would be an easy > change to make). ?If you want to use a multiplicative relaxation like SOR, > you would have to implement it yourself. ?If you need something like ILU for > a smoother, then you will have to pay for a matrix. ?Note that one > possibility is to assemble a matrix on all but the finest level so you can > use stronger smoothers there, then make do with Jacobi on the finest level. > >> >> 3) to deal with non-uniform sampling: can I provide the coordinates of >> the finest grid with DASetCoordinates, then expect DMMG to provide the >> subsampled coordinates at the coarser levels? > > Currently no, you have to set them on each level. ?Perhaps you could do this > by rolling a loop over levels?and applying MatRestrict using the restriction > matrix from DMMG (this might not be the sampling that you want). > Jed -------------- next part -------------- A non-text attachment was scrubbed... Name: my_green.f90 Type: application/octet-stream Size: 29788 bytes Desc: not available URL: From bsmith at mcs.anl.gov Tue May 10 22:05:46 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 May 2011 22:05:46 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <0B5B1110339742BC8990AB61531039A8@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: Ok, the linear solver looks like it is working ok. The likely problem is that Jacobian does not match the function evaluation. Run the same thing but with the additional option -snes_mf_operator Then run with -snes_type test (instead of -snes_mf_operator). Barry On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: > Dear Barry, here is the output using -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor > the attached is the same and for clear reference. Thanks again for helps. > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP Residual norm 9.965778978387e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.823187455811e-017 > 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 > 1 SNES Function norm 6.401926523423e-002 > 0 KSP Residual norm 8.969200212486e-002 > 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.106757475780e-016 > 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 > 2 SNES Function norm 5.849992149767e-002 > 0 KSP Residual norm 8.072279488157e-002 > 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.268750073799e-017 > 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 > 3 SNES Function norm 5.376618503592e-002 > 0 KSP Residual norm 7.265050969883e-002 > 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.677655733356e-017 > 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 > 4 SNES Function norm 4.956894354459e-002 > 0 KSP Residual norm 6.538545411661e-002 > 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.557004153175e-017 > 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 > 5 SNES Function norm 4.575418613137e-002 > 0 KSP Residual norm 5.884690496914e-002 > 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.470969262115e-017 > 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 > 6 SNES Function norm 4.223022245585e-002 > 0 KSP Residual norm 5.296221144636e-002 > 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.255198782390e-017 > 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 > 7 SNES Function norm 3.894430065910e-002 > 0 KSP Residual norm 4.766598785088e-002 > 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.322615478395e-017 > 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 > 8 SNES Function norm 3.586683371135e-002 > 0 KSP Residual norm 4.289938708067e-002 > 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.181358328498e-017 > 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 > 9 SNES Function norm 3.298130202025e-002 > 0 KSP Residual norm 3.860944676473e-002 > 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.635174776374e-017 > 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 > 10 SNES Function norm 3.027806208930e-002 > 0 KSP Residual norm 3.474850078591e-002 > 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.389914053685e-017 > 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 > 11 SNES Function norm 2.749422924729e-002 > 0 KSP Residual norm 3.081350823297e-002 > 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.640567497647e-017 > 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 > 12 SNES Function norm 2.437488247885e-002 > 0 KSP Residual norm 2.633007441879e-002 > 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.772331460094e-017 > 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 > 13 SNES Function norm 2.079664278637e-002 > 0 KSP Residual norm 2.104738289397e-002 > 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.650632708670e-017 > 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 > 14 SNES Function norm 1.657344626858e-002 > 0 KSP Residual norm 1.454141853505e-002 > 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.129401160070e-017 > 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 > 15 SNES Function norm 1.484243752612e-002 > 0 KSP Residual norm 5.241948491751e-009 > 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.729506849025e-024 > 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 > 16 SNES Function norm 2.828002157497e-008 > 0 KSP Residual norm 6.042518362322e-015 > 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.272441346127e-030 > 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 > 17 SNES Function norm 2.960967020289e-008 > STEP 0 (Newton iterations: 17) > > diverged reason: -6 > > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 2:54 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and send the outputs > > > Barry > > On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: > >> by the way, the increment size is like that >> for a 100 lengh model, the increment is set to 0.05, >> the engineering strain is around 5%% per load step. >> This is already too small increment size for a large deformation analysis. >> a 0.5 increment size leads to both linear search and trust region failed. >> linear search failed for 0.05 while trust region converges with 17 Newton iterations each load step. >> Rong >> >> ----- Original Message ----- From: "Tian(ICT)" >> To: "PETSc users list" >> Sent: Tuesday, May 10, 2011 11:37 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >>> First, thanks again, the issue was gone. >>> >>> I just followed up with some test results. >>> I have tested SNES using one finite element for a geometric large deformation problem. >>> Those are just the very early test results so they may be not telling what happened exactly. >>> For the displacement controlled load, I found that convergence is much slower than that of force loading. >>> Even worse, linear search is so sensitive to the displacement increment and diverged no matter what the increment size was used (too small incremnt also led to diverged soloution (-6 reason), trust region works well in the sense of not sensitive to the displacement increment, but during each load step, it requires around ten to several tens of Newton interations whereas for the force loading case and the almost same amount of deformation, this is normally 3. This is against my expectation. Any hint? >>> >>> Rong >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 10:22 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>> >>>> Dear Barry, Thanks a lot for quick answering. >>>> I checked the development documents and found the new version of MatZeroRows() does support the nonzero prescribed boundary conditions. >>>> >>>> I followed up with more details. >>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES solvers. >>>> I used a displacement-controlled load (as this type of loading works well for all cases). >>>> This is the reason the nonzero prescribed boundary came up. >>>> >>>> In FormJacobian, I modified Jacobian and residual to satisfy the nonzero prescribed boundary. >>>> In FormFunction, I modified the solution to the known solution(this should not be necessary as the modified Jacobian and rhs should give the prescribed solution also) >>> >>> You should not do it this way. See below. >>>> >>>> Now I found another issue, no matter if I prescried the solution or not in FormFunction, >>>> SNES solver always call FormFunction and never call FormJacobian. >>> >>> The only reason it would not call FormJacobian is if decided that the residual norm was small enough before any Newton steps; for example if the FormFunction() computed exactly the zero function initially. When you run with -snes_monitor -ksp_monitor what does it print for residual norms. >>> >>>> Of course the solver finally diverged or converged to a zero solution. >>>> >>>> So my quick follow up question is How a displacement-controled load is done corrently in Petsc 2.3.3? >>> >>> To do it in 2.3.3 simply have for those components of F() the formula F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for those rows >>> >>> We strongly urge you to upgrade to the latest PETSc before doing anything further. >>> >>> >>> Barry >>> >>>> >>>> Rong >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> In petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have modified the calling sequence for MatZeroRows() so that it can automatically adjust the appropriate right hand side values for the zeroed rows to support zero or non-zero prescribed boundary conditions easily. >>>> >>>> Barry >>>> >>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>> >>>>> Dear all, >>>>> >>>>> I got this question long ago and searched the prior posting but did not find the solution. >>>>> The question is about nonzero prescribed boundary condition. >>>>> My understanding is that MatZeroRows() works only for zero prescribed value, not non-zero value. >>>>> For the non-zero values, we have to remove the rows associated with the boundary, but this >>>>> will lead to a zero dignal and accordingly the rows in r.h.s should also be removed. >>>>> My question is that does MatZeroRows() also works for nonzero prescribed boundary and if so how to do it simply? >>>>> >>>>> Rong >>>> >>>> >>> >>> >> > > From bsmith at mcs.anl.gov Tue May 10 22:33:38 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 May 2011 22:33:38 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: Barry On May 10, 2011, at 9:20 PM, Sylvain Barbot wrote: > Dear Jed, > > Thank you for your previous comments. I am still trying to design a > multigrid preconditionner for the Navier's equation of elasticity. I > have defined shell matrices for the interpolation, restriction, > smoothing, residual calculation and the fine-level solver (see > attached my_green.f90 file with stub functions). I thought that all > these shell matrices would be enough to do the job, but upon running > KSPSetup(), I get the following error: > > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: This matrix type, shell, does not support getting > diagonal block! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 > 14:42:02 CDT 2010 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c > > regarding your previous comments: "For Jacobi, you can have your > MatShell implement MatGetDiagonal and it will work. (...) If you want > to use a multiplicative relaxation like SOR, you would have to > implement it yourself.", I actually want to use SOR with my smoother > shell matrix. What am I doing wrong to make petsc still believe I want > to use Jacobi? > By default, it is using block Jacobi on each level (except the coarsest). You need to tell it to use the relaxation you have coded. For example if your shell matrix provides a SOR/relaxation method then you need to use -mg_levels_pc_type sor and it will call that on each level instead of block Jacobi. In looking at your code I see the only operation your provide your shell matrices are matrix-vector multiply. Therefore you cannot use SOR/relaxation. The only think you can use is GMRES (or some other Krylov method) with no preconditioner as the smoother. For this you can use -mg_levels_pc_type none -mg_levels_ksp_type gmres In your message you say "I actually want to use SOR with my smoother" to do this you need to provide the smoother function and set it with MatShellSetOperations(). > another related question: with a direct solver, one needs to assign a > shell matrix with a KSP object (KSPSetOperators); is it still required > when using multigrid? if I don't do so, I get the following error: > Where are you using the direct solver for, on the entire linear system or on the coarse mesh? In either case you NEED to provide an explicit (not shell) sparse matrix otherwise a direct solver cannot be used. From the message below it looks like you did not provide a matrix somewhere. If you don't provide a matrix a direct solver cannot be used. Barry > [0]PETSC ERROR: Null argument, when expecting valid pointer! > [0]PETSC ERROR: Null Object: Parameter # 1! > [0]PETSC ERROR: MatGetVecs() line 7265 in src/mat/interface/matrix.c > [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c > [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: KSPSetUp() line 199 in src/ksp/ksp/interface/itfunc.c > > however, I don't understand why I would need this matrix in a > multigrid framework. > > Is there any example out there of using PCMG with shell matrices? > > Thanks in advance for your response. > Best wishes, > Sylvain Barbot > > > 2010/10/7 Jed Brown : >> On Thu, Oct 7, 2010 at 11:45, Sylvain Barbot >> wrote: >>> >>> questions: >>> 1) is it at all possible to specify this mode of operation, from the >>> finest to the coarser level, and back? Any examples out there? >> >> -pc_mg_type multiplicative, or >> PCMGSetType(pc,PC_MG_MULTIPLICATIVE); >> >>> >>> 2) is it readily possible to use matrix shells with DMMG? I imagine >>> the Jacobian matrix may simply be provided as a matrix shell. Is there >>> any examples of multi-grid methods with shell matrices online? >> >> You can do this, but you will have to define a custom smoother. For Jacobi, >> you can have your MatShell implement MatGetDiagonal and it will work. I >> thought you could implement MatGetDiagonalBlock for PBJacobi, but it's not >> currently written that way (though it should be and that would be an easy >> change to make). If you want to use a multiplicative relaxation like SOR, >> you would have to implement it yourself. If you need something like ILU for >> a smoother, then you will have to pay for a matrix. Note that one >> possibility is to assemble a matrix on all but the finest level so you can >> use stronger smoothers there, then make do with Jacobi on the finest level. >> >>> >>> 3) to deal with non-uniform sampling: can I provide the coordinates of >>> the finest grid with DASetCoordinates, then expect DMMG to provide the >>> subsampled coordinates at the coarser levels? >> >> Currently no, you have to set them on each level. Perhaps you could do this >> by rolling a loop over levels and applying MatRestrict using the restriction >> matrix from DMMG (this might not be the sampling that you want). >> Jed > From rongtian at ncic.ac.cn Wed May 11 01:16:57 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Wed, 11 May 2011 14:16:57 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: With additional -snes_type test: "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 Testing hand-coded Jacobian, if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct. Run with -snes_test_display to show difference of hand-coded and finite difference Jacobian. Norm of matrix ratio 9.05702e-008 difference 3.94971 Norm of matrix ratio 3.6015e-007 difference 15.7059 Norm of matrix ratio 3.73941e-007 difference 16.3073 [0]PETSC ERROR: SNESSolve() line 1871 in src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c [0]PETSC ERROR: User provided function() line 572 in unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp [0]PETSC ERROR: User provided function() line 40 in unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp -snes_mf_operator does not work: "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Object is in wrong state! [0]PETSC ERROR: Not for unassembled matrix! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 15, Tue Sep 23 10:02:49 CDT 2008 HG revision: 31306062cd1a6f6a2496fccb4878f485c9b91760 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: pgfem on a cygwin-gc named RTI458-LAPTOP by rabbit Wed May 11 14:16:08 2011 [0]PETSC ERROR: Libraries linked from /home/petsc-2.3.3-p15/lib/cygwin-gcc [0]PETSC ERROR: Configure run at Wed Dec 1 17:02:20 2010 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-cxx="win32fe cl" --download-c-blas-lapack=1 --with-clanguage=cxx --with-fc=0 --useThreads=0 --with-shared=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatDuplicate() line 3218 in src/mat/interface/C:\cygwin\home\PETSC-~1.3-P\src\mat\INTERF~1\matrix.c atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067811865e-002 0 KSP Residual norm 9.965778978387e-002 [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Null argument, when expecting valid pointer! [0]PETSC ERROR: Null Object: Parameter # 1! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 15, Tue Sep 23 10:02:49 CDT 2008 HG revision: 31306062cd1a6f6a2496fccb4878f485c9b91760 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: pgfem on a cygwin-gc named RTI458-LAPTOP by rabbit Wed May 11 14:16:08 2011 [0]PETSC ERROR: Libraries linked from /home/petsc-2.3.3-p15/lib/cygwin-gcc [0]PETSC ERROR: Configure run at Wed Dec 1 17:02:20 2010 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-cxx="win32fe cl" --download-c-blas-lapack=1 --with-clanguage=cxx --with-fc=0 --useThreads=0 --with-shared=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PetscObjectGetComm() line 34 in src/sys/objects/C:\cygwin\home\PETSC-~1.3-P\src\sys\objects\gcomm.c [0]PETSC ERROR: VecNormBegin() line 495 in src/vec/vec/utils/C:\cygwin\home\PETSC-~1.3-P\src\vec\vec\utils\comb.c [0]PETSC ERROR: MatMFFDCompute_WP() line 73 in src/mat/impls/mffd/C:\cygwin\home\PETSC-~1.3-P\src\mat\impls\mffd\wp.c [0]PETSC ERROR: MatMult_MFFD() line 294 in src/mat/impls/mffd/C:\cygwin\home\PETSC-~1.3-P\src\mat\impls\mffd\mffd.c [0]PETSC ERROR: MatMult() line 1632 in src/mat/interface/C:\cygwin\home\PETSC-~1.3-P\src\mat\INTERF~1\matrix.c [0]PETSC ERROR: KSPDefaultBuildResidual() line 605 in src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\iterativ.c [0]PETSC ERROR: KSPBuildResidual() line 1587 in src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\itfunc.c [0]PETSC ERROR: KSPMonitorTrueResidualNorm() line 258 in src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\iterativ.c [0]PETSC ERROR: GMREScycle() line 155 in src/ksp/ksp/impls/gmres/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\impls\gmres\gmres.c [0]PETSC ERROR: KSPSolve_GMRES() line 241 in src/ksp/ksp/impls/gmres/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\impls\gmres\gmres.c [0]PETSC ERROR: KSPSolve() line 379 in src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\itfunc.c [0]PETSC ERROR: SNES_KSPSolve() line 2578 in src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c [0]PETSC ERROR: SNESSolve_LS() line 184 in src/snes/impls/ls/C:\cygwin\home\PETSC-~1.3-P\src\snes\impls\ls\ls.c [0]PETSC ERROR: SNESSolve() line 1871 in src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c [0]PETSC ERROR: User provided function() line 574 in unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp [0]PETSC ERROR: User provided function() line 40 in unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp Rong ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Wednesday, May 11, 2011 11:05 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition Ok, the linear solver looks like it is working ok. The likely problem is that Jacobian does not match the function evaluation. Run the same thing but with the additional option -snes_mf_operator Then run with -snes_type test (instead of -snes_mf_operator). Barry On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: > Dear Barry, here is the output using -pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor > the attached is the same and for clear reference. Thanks again for helps. > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP Residual norm 9.965778978387e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm > 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.823187455811e-017 > 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm > 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 > 1 SNES Function norm 6.401926523423e-002 > 0 KSP Residual norm 8.969200212486e-002 > 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm > 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.106757475780e-016 > 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm > 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 > 2 SNES Function norm 5.849992149767e-002 > 0 KSP Residual norm 8.072279488157e-002 > 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm > 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.268750073799e-017 > 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm > 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 > 3 SNES Function norm 5.376618503592e-002 > 0 KSP Residual norm 7.265050969883e-002 > 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm > 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.677655733356e-017 > 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm > 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 > 4 SNES Function norm 4.956894354459e-002 > 0 KSP Residual norm 6.538545411661e-002 > 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm > 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.557004153175e-017 > 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm > 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 > 5 SNES Function norm 4.575418613137e-002 > 0 KSP Residual norm 5.884690496914e-002 > 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm > 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.470969262115e-017 > 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm > 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 > 6 SNES Function norm 4.223022245585e-002 > 0 KSP Residual norm 5.296221144636e-002 > 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm > 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.255198782390e-017 > 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm > 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 > 7 SNES Function norm 3.894430065910e-002 > 0 KSP Residual norm 4.766598785088e-002 > 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm > 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.322615478395e-017 > 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm > 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 > 8 SNES Function norm 3.586683371135e-002 > 0 KSP Residual norm 4.289938708067e-002 > 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm > 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.181358328498e-017 > 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm > 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 > 9 SNES Function norm 3.298130202025e-002 > 0 KSP Residual norm 3.860944676473e-002 > 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm > 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.635174776374e-017 > 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm > 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 > 10 SNES Function norm 3.027806208930e-002 > 0 KSP Residual norm 3.474850078591e-002 > 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm > 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.389914053685e-017 > 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm > 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 > 11 SNES Function norm 2.749422924729e-002 > 0 KSP Residual norm 3.081350823297e-002 > 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm > 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.640567497647e-017 > 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm > 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 > 12 SNES Function norm 2.437488247885e-002 > 0 KSP Residual norm 2.633007441879e-002 > 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm > 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.772331460094e-017 > 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm > 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 > 13 SNES Function norm 2.079664278637e-002 > 0 KSP Residual norm 2.104738289397e-002 > 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm > 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.650632708670e-017 > 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm > 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 > 14 SNES Function norm 1.657344626858e-002 > 0 KSP Residual norm 1.454141853505e-002 > 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm > 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.129401160070e-017 > 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm > 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 > 15 SNES Function norm 1.484243752612e-002 > 0 KSP Residual norm 5.241948491751e-009 > 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm > 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.729506849025e-024 > 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm > 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 > 16 SNES Function norm 2.828002157497e-008 > 0 KSP Residual norm 6.042518362322e-015 > 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm > 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.272441346127e-030 > 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm > 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 > 17 SNES Function norm 2.960967020289e-008 > STEP 0 (Newton iterations: 17) > > diverged reason: -6 > > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 2:54 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and > send the outputs > > > Barry > > On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: > >> by the way, the increment size is like that >> for a 100 lengh model, the increment is set to 0.05, >> the engineering strain is around 5%% per load step. >> This is already too small increment size for a large deformation >> analysis. >> a 0.5 increment size leads to both linear search and trust region failed. >> linear search failed for 0.05 while trust region converges with 17 Newton >> iterations each load step. >> Rong >> >> ----- Original Message ----- From: "Tian(ICT)" >> To: "PETSc users list" >> Sent: Tuesday, May 10, 2011 11:37 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >>> First, thanks again, the issue was gone. >>> >>> I just followed up with some test results. >>> I have tested SNES using one finite element for a geometric large >>> deformation problem. >>> Those are just the very early test results so they may be not telling >>> what happened exactly. >>> For the displacement controlled load, I found that convergence is much >>> slower than that of force loading. >>> Even worse, linear search is so sensitive to the displacement increment >>> and diverged no matter what the increment size was used (too small >>> incremnt also led to diverged soloution (-6 reason), trust region works >>> well in the sense of not sensitive to the displacement increment, but >>> during each load step, it requires around ten to several tens of Newton >>> interations whereas for the force loading case and the almost same >>> amount of deformation, this is normally 3. This is against my >>> expectation. Any hint? >>> >>> Rong >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 10:22 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>> >>>> Dear Barry, Thanks a lot for quick answering. >>>> I checked the development documents and found the new version of >>>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>>> >>>> I followed up with more details. >>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>>> solvers. >>>> I used a displacement-controlled load (as this type of loading works >>>> well for all cases). >>>> This is the reason the nonzero prescribed boundary came up. >>>> >>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>> nonzero prescribed boundary. >>>> In FormFunction, I modified the solution to the known solution(this >>>> should not be necessary as the modified Jacobian and rhs should give >>>> the prescribed solution also) >>> >>> You should not do it this way. See below. >>>> >>>> Now I found another issue, no matter if I prescried the solution or not >>>> in FormFunction, >>>> SNES solver always call FormFunction and never call FormJacobian. >>> >>> The only reason it would not call FormJacobian is if decided that the >>> residual norm was small enough before any Newton steps; for example if >>> the FormFunction() computed exactly the zero function initially. When >>> you run with -snes_monitor -ksp_monitor what does it print for residual >>> norms. >>> >>>> Of course the solver finally diverged or converged to a zero solution. >>>> >>>> So my quick follow up question is How a displacement-controled load is >>>> done corrently in Petsc 2.3.3? >>> >>> To do it in 2.3.3 simply have for those components of F() the formula >>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for >>> those rows >>> >>> We strongly urge you to upgrade to the latest PETSc before doing >>> anything further. >>> >>> >>> Barry >>> >>>> >>>> Rong >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> In petsc-dev >>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>> modified the calling sequence for MatZeroRows() so that it can >>>> automatically adjust the appropriate right hand side values for the >>>> zeroed rows to support zero or non-zero prescribed boundary conditions >>>> easily. >>>> >>>> Barry >>>> >>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>> >>>>> Dear all, >>>>> >>>>> I got this question long ago and searched the prior posting but did >>>>> not find the solution. >>>>> The question is about nonzero prescribed boundary condition. >>>>> My understanding is that MatZeroRows() works only for zero prescribed >>>>> value, not non-zero value. >>>>> For the non-zero values, we have to remove the rows associated with >>>>> the boundary, but this >>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>> also be removed. >>>>> My question is that does MatZeroRows() also works for nonzero >>>>> prescribed boundary and if so how to do it simply? >>>>> >>>>> Rong >>>> >>>> >>> >>> >> > > From jed at 59A2.org Wed May 11 01:33:47 2011 From: jed at 59A2.org (Jed Brown) Date: Wed, 11 May 2011 08:33:47 +0200 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: On Wed, May 11, 2011 at 08:16, Tian(ICT) wrote: > -snes_mf_operator does not work: > > "-pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" > I assume you spelled that -snes_mf_operator (not -snes_ms_operator). Anyway, I think the problem is that the MFFD operator is not "assembled". At the bottom of your FormJacobian(), you should have something like: ierr = MatAssemblyBegin(*B,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(*B,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); if (*A != *B) { ierr = MatAssemblyBegin(*A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); ierr = MatAssemblyEnd(*A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); } The second line resets the base vector when the Krylov operator A is different from the preconditioning matrix B. When you run with -snes_mf_operator, SNES internally changes A to be an MFFD matrix, and your original matrix is used for B, therefore that is the matrix you should call MatSetValues() on. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Wed May 11 01:46:53 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Wed, 11 May 2011 14:46:53 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: <1302D684D3644298A3F2528BDEE08857@rti458laptop> Pls ignore my previous email. The followings are the test results: the previous error in using -snes_mf_opertor was due to my own code. After modification, i.e. using different A and B and assemble both and in FormJacobian the native jacobian was assembled into B. Then with -snes_mf_operator led to the results shown below. The first three steps converged but with a diverged reason -6 at step 4. With additional -snes_type test, the results were > "-pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > Testing hand-coded Jacobian, if the ratio is > O(1.e-8), the hand-coded Jacobian is probably correct. > Run with -snes_test_display to show difference > of hand-coded and finite difference Jacobian. > Norm of matrix ratio 9.05702e-008 difference 3.94971 > Norm of matrix ratio 3.6015e-007 difference 15.7059 > Norm of matrix ratio 3.73941e-007 difference 16.3073 > [0]PETSC ERROR: SNESSolve() line 1871 in > src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c > [0]PETSC ERROR: User provided function() line 572 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp > [0]PETSC ERROR: User provided function() line 40 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp > > > "-pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067811865e-002 0 KSP Residual norm 9.965778978387e-002 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 9.102368266281e-009 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 1 SNES Function norm 6.401926196865e-002 0 KSP Residual norm 8.969200166800e-002 0 KSP preconditioned resid norm 8.969200166800e-002 true resid norm 6.401926196865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.492849351675e-009 1 KSP preconditioned resid norm 8.492849351675e-009 true resid norm 1.400035213572e-002 ||Ae||/||Ax|| 2.186896834670e-001 2 SNES Function norm 5.849991037388e-002 0 KSP Residual norm 8.072279482667e-002 0 KSP preconditioned resid norm 8.072279482667e-002 true resid norm 5.849991037388e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.610510067555e-009 1 KSP preconditioned resid norm 3.610510067555e-009 true resid norm 4.757549946973e-003 ||Ae||/||Ax|| 8.132576471599e-002 3 SNES Function norm 5.376617142827e-002 0 KSP Residual norm 7.265050986128e-002 0 KSP preconditioned resid norm 7.265050986128e-002 true resid norm 5.376617142827e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 6.941165647221e-009 1 KSP preconditioned resid norm 6.941165647221e-009 true resid norm 3.935270526562e-002 ||Ae||/||Ax|| 7.319231446138e-001 4 SNES Function norm 4.956888929193e-002 0 KSP Residual norm 6.538545408372e-002 0 KSP preconditioned resid norm 6.538545408372e-002 true resid norm 4.956888929193e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.457970649220e-009 1 KSP preconditioned resid norm 3.457970649220e-009 true resid norm 2.037700110050e-002 ||Ae||/||Ax|| 4.110844804388e-001 5 SNES Function norm 4.575415001780e-002 0 KSP Residual norm 5.884690527647e-002 0 KSP preconditioned resid norm 5.884690527647e-002 true resid norm 4.575415001780e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.832662187541e-009 1 KSP preconditioned resid norm 2.832662187541e-009 true resid norm 1.662420124938e-002 ||Ae||/||Ax|| 3.633375604818e-001 6 SNES Function norm 4.223018337199e-002 0 KSP Residual norm 5.296221162333e-002 0 KSP preconditioned resid norm 5.296221162333e-002 true resid norm 4.223018337199e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.091667005099e-009 1 KSP preconditioned resid norm 3.091667005099e-009 true resid norm 2.235399010718e-002 ||Ae||/||Ax|| 5.293367994705e-001 7 SNES Function norm 3.894426990170e-002 0 KSP Residual norm 4.766598828506e-002 0 KSP preconditioned resid norm 4.766598828506e-002 true resid norm 3.894426990170e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.223811830585e-009 1 KSP preconditioned resid norm 4.223811830585e-009 true resid norm 2.831156780040e-002 ||Ae||/||Ax|| 7.269764684732e-001 8 SNES Function norm 3.586679674893e-002 0 KSP Residual norm 4.289938722959e-002 0 KSP preconditioned resid norm 4.289938722959e-002 true resid norm 3.586679674893e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.504535460429e-009 1 KSP preconditioned resid norm 3.504535460429e-009 true resid norm 2.254598873838e-002 ||Ae||/||Ax|| 6.286033541328e-001 9 SNES Function norm 3.298126871313e-002 0 KSP Residual norm 3.860944657267e-002 0 KSP preconditioned resid norm 3.860944657267e-002 true resid norm 3.298126871313e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.439084585574e-009 1 KSP preconditioned resid norm 4.439084585574e-009 true resid norm 6.936792391193e-003 ||Ae||/||Ax|| 2.103252137305e-001 10 SNES Function norm 3.027806037514e-002 0 KSP Residual norm 3.474850031738e-002 0 KSP preconditioned resid norm 3.474850031738e-002 true resid norm 3.027806037514e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.476854671229e-009 1 KSP preconditioned resid norm 3.476854671229e-009 true resid norm 1.935717696630e-002 ||Ae||/||Ax|| 6.393136391984e-001 11 SNES Function norm 2.775064646763e-002 0 KSP Residual norm 3.127364888698e-002 0 KSP preconditioned resid norm 3.127364888698e-002 true resid norm 2.775064646763e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.380857715198e-009 1 KSP preconditioned resid norm 2.380857715198e-009 true resid norm 1.646856223644e-002 ||Ae||/||Ax|| 5.934478771746e-001 12 SNES Function norm 2.430399371900e-002 0 KSP Residual norm 2.602654480862e-002 0 KSP preconditioned resid norm 2.602654480862e-002 true resid norm 2.430399371900e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.556142726483e-009 1 KSP preconditioned resid norm 2.556142726483e-009 true resid norm 1.794651743617e-002 ||Ae||/||Ax|| 7.384184526900e-001 13 SNES Function norm 2.029256664631e-002 0 KSP Residual norm 1.954645630691e-002 0 KSP preconditioned resid norm 1.954645630691e-002 true resid norm 2.029256664631e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.913585412092e-010 1 KSP preconditioned resid norm 3.913585412092e-010 true resid norm 1.903675728342e-003 ||Ae||/||Ax|| 9.381148089946e-002 14 SNES Function norm 1.566638821565e-002 0 KSP Residual norm 1.231404128086e-002 0 KSP preconditioned resid norm 1.231404128086e-002 true resid norm 1.566638821565e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 7.258598971379e-010 1 KSP preconditioned resid norm 7.258598971379e-010 true resid norm 1.221317129610e-003 ||Ae||/||Ax|| 7.795779810881e-002 15 SNES Function norm 1.064422587384e-002 0 KSP Residual norm 3.723145759360e-009 0 KSP preconditioned resid norm 3.723145759360e-009 true resid norm 1.064422587384e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.546825086207e-016 1 KSP preconditioned resid norm 5.546825086207e-016 true resid norm 2.984647735625e-009 ||Ae||/||Ax|| 2.804006389003e-007 16 SNES Function norm 9.689226075617e-008 0 KSP Residual norm 1.643080767133e-014 0 KSP preconditioned resid norm 1.643080767133e-014 true resid norm 9.689226075617e-008 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 7.291965935374e-022 1 KSP preconditioned resid norm 7.291965935374e-022 true resid norm 5.001850432973e-015 ||Ae||/||Ax|| 5.162280654758e-008 17 SNES Function norm 3.747290584108e-008 STEP 0 (Newton iterations: 17) 0 SNES Function norm 7.071067811866e-002 0 KSP Residual norm 9.965769844003e-002 0 KSP preconditioned resid norm 9.965769844003e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.919247724727e-009 1 KSP preconditioned resid norm 4.919247724727e-009 true resid norm 2.073848008334e-002 ||Ae||/||Ax|| 2.932863979685e-001 1 SNES Function norm 6.402041310994e-002 0 KSP Residual norm 8.969192038029e-002 0 KSP preconditioned resid norm 8.969192038029e-002 true resid norm 6.402041310994e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.072962844649e-009 1 KSP preconditioned resid norm 4.072962844649e-009 true resid norm 2.116458841832e-002 ||Ae||/||Ax|| 3.305912503559e-001 2 SNES Function norm 5.850358277995e-002 0 KSP Residual norm 8.072272129834e-002 0 KSP preconditioned resid norm 8.072272129834e-002 true resid norm 5.850358277995e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.535693836801e-009 1 KSP preconditioned resid norm 1.535693836801e-009 true resid norm 1.378413836547e-002 ||Ae||/||Ax|| 2.356118670085e-001 3 SNES Function norm 5.377276379928e-002 0 KSP Residual norm 7.265044334812e-002 0 KSP preconditioned resid norm 7.265044334812e-002 true resid norm 5.377276379928e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.196914267894e-009 1 KSP preconditioned resid norm 8.196914267894e-009 true resid norm 1.643681592299e-002 ||Ae||/||Ax|| 3.056717706448e-001 4 SNES Function norm 4.957825627064e-002 0 KSP Residual norm 6.538539498475e-002 0 KSP preconditioned resid norm 6.538539498475e-002 true resid norm 4.957825627064e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.605278143372e-009 1 KSP preconditioned resid norm 1.605278143372e-009 true resid norm 1.365749144340e-002 ||Ae||/||Ax|| 2.754734125550e-001 5 SNES Function norm 4.576573391352e-002 0 KSP Residual norm 5.884685187863e-002 0 KSP preconditioned resid norm 5.884685187863e-002 true resid norm 4.576573391352e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.917790130377e-009 1 KSP preconditioned resid norm 3.917790130377e-009 true resid norm 2.918137698804e-002 ||Ae||/||Ax|| 6.376250196967e-001 6 SNES Function norm 4.224348383506e-002 0 KSP Residual norm 5.296216341013e-002 0 KSP preconditioned resid norm 5.296216341013e-002 true resid norm 4.224348383506e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.384279752582e-009 1 KSP preconditioned resid norm 1.384279752582e-009 true resid norm 8.645143617596e-003 ||Ae||/||Ax|| 2.046503468168e-001 7 SNES Function norm 3.895875122953e-002 0 KSP Residual norm 4.766594475257e-002 0 KSP preconditioned resid norm 4.766594475257e-002 true resid norm 3.895875122953e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.442999105246e-009 1 KSP preconditioned resid norm 5.442999105246e-009 true resid norm 3.199366262059e-002 ||Ae||/||Ax|| 8.212188946227e-001 8 SNES Function norm 3.588203219083e-002 0 KSP Residual norm 4.289934859308e-002 0 KSP preconditioned resid norm 4.289934859308e-002 true resid norm 3.588203219083e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.069272581944e-009 1 KSP preconditioned resid norm 5.069272581944e-009 true resid norm 3.315026802067e-002 ||Ae||/||Ax|| 9.238681868510e-001 9 SNES Function norm 3.299674865285e-002 0 KSP Residual norm 3.860941169595e-002 0 KSP preconditioned resid norm 3.860941169595e-002 true resid norm 3.299674865285e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.314510318621e-009 1 KSP preconditioned resid norm 2.314510318621e-009 true resid norm 1.422355468145e-002 ||Ae||/||Ax|| 4.310592789335e-001 10 SNES Function norm 3.020092421335e-002 0 KSP Residual norm 3.457428537321e-002 0 KSP preconditioned resid norm 3.457428537321e-002 true resid norm 3.020092421335e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.171225245965e-009 1 KSP preconditioned resid norm 3.171225245965e-009 true resid norm 6.700877909403e-003 ||Ae||/||Ax|| 2.218765843742e-001 11 SNES Function norm 2.761432326644e-002 0 KSP Residual norm 3.100402216444e-002 0 KSP preconditioned resid norm 3.100402216444e-002 true resid norm 2.761432326644e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.317753799744e-009 1 KSP preconditioned resid norm 2.317753799744e-009 true resid norm 1.143024510909e-002 ||Ae||/||Ax|| 4.139245057287e-001 12 SNES Function norm 2.483228477543e-002 0 KSP Residual norm 2.716039584199e-002 0 KSP preconditioned resid norm 2.716039584199e-002 true resid norm 2.483228477543e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.024860175617e-009 1 KSP preconditioned resid norm 1.024860175617e-009 true resid norm 5.071961436716e-003 ||Ae||/||Ax|| 2.042486820115e-001 13 SNES Function norm 2.155599009420e-002 0 KSP Residual norm 2.247739022940e-002 0 KSP preconditioned resid norm 2.247739022940e-002 true resid norm 2.155599009420e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.231896568498e-009 1 KSP preconditioned resid norm 1.231896568498e-009 true resid norm 2.253717888475e-003 ||Ae||/||Ax|| 1.045518150002e-001 14 SNES Function norm 1.758159262430e-002 0 KSP Residual norm 1.640654408633e-002 0 KSP preconditioned resid norm 1.640654408633e-002 true resid norm 1.758159262430e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.203409491551e-009 1 KSP preconditioned resid norm 1.203409491551e-009 true resid norm 3.340286893989e-003 ||Ae||/||Ax|| 1.899877312236e-001 15 SNES Function norm 1.275874713861e-002 0 KSP Residual norm 8.737890404296e-003 0 KSP preconditioned resid norm 8.737890404296e-003 true resid norm 1.275874713861e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.862616851249e-010 1 KSP preconditioned resid norm 3.862616851249e-010 true resid norm 9.158071400861e-004 ||Ae||/||Ax|| 7.177876715771e-002 16 SNES Function norm 5.367994969686e-003 0 KSP Residual norm 1.873266786098e-009 0 KSP preconditioned resid norm 1.873266786098e-009 true resid norm 5.367994969686e-003 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 9.236612780542e-017 1 KSP preconditioned resid norm 9.236612780542e-017 true resid norm 4.487656305545e-010 ||Ae||/||Ax|| 8.360023306445e-008 17 SNES Function norm 1.843858578272e-008 0 KSP Residual norm 3.418191528615e-015 0 KSP preconditioned resid norm 3.418191528615e-015 true resid norm 1.843858578272e-008 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.091028447697e-022 1 KSP preconditioned resid norm 2.091028447697e-022 true resid norm 9.356082852780e-016 ||Ae||/||Ax|| 5.074186796662e-008 18 SNES Function norm 1.582175998776e-008 STEP 1 (Newton iterations: 18) 0 SNES Function norm 7.071067811866e-002 0 KSP Residual norm 9.965760716669e-002 0 KSP preconditioned resid norm 9.965760716669e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.158690871520e-009 1 KSP preconditioned resid norm 4.158690871520e-009 true resid norm 1.356168089543e-002 ||Ae||/||Ax|| 1.917911305090e-001 1 SNES Function norm 6.402155946870e-002 0 KSP Residual norm 8.969183775926e-002 0 KSP preconditioned resid norm 8.969183775926e-002 true resid norm 6.402155946870e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 7.922035131300e-009 1 KSP preconditioned resid norm 7.922035131300e-009 true resid norm 4.173737132742e-002 ||Ae||/||Ax|| 6.519268145573e-001 2 SNES Function norm 5.850726192206e-002 0 KSP Residual norm 8.072264774465e-002 0 KSP preconditioned resid norm 8.072264774465e-002 true resid norm 5.850726192206e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.356474211542e-009 1 KSP preconditioned resid norm 4.356474211542e-009 true resid norm 2.083500604436e-002 ||Ae||/||Ax|| 3.561097436437e-001 3 SNES Function norm 5.377933295280e-002 0 KSP Residual norm 7.265037686612e-002 0 KSP preconditioned resid norm 7.265037686612e-002 true resid norm 5.377933295280e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.772318530630e-009 1 KSP preconditioned resid norm 8.772318530630e-009 true resid norm 5.284958913103e-002 ||Ae||/||Ax|| 9.827118751623e-001 4 SNES Function norm 4.958751191600e-002 0 KSP Residual norm 6.538533386375e-002 0 KSP preconditioned resid norm 6.538533386375e-002 true resid norm 4.958751191600e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.128741903850e-009 1 KSP preconditioned resid norm 1.128741903850e-009 true resid norm 2.526410501148e-003 ||Ae||/||Ax|| 5.094852319729e-002 5 SNES Function norm 4.577729047509e-002 0 KSP Residual norm 5.884679667348e-002 0 KSP preconditioned resid norm 5.884679667348e-002 true resid norm 4.577729047509e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.763627011382e-009 1 KSP preconditioned resid norm 5.763627011382e-009 true resid norm 1.051243252216e-002 ||Ae||/||Ax|| 2.296429608013e-001 6 SNES Function norm 4.225677503639e-002 0 KSP Residual norm 5.296211439112e-002 0 KSP preconditioned resid norm 5.296211439112e-002 true resid norm 4.225677503639e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.022912340341e-009 1 KSP preconditioned resid norm 1.022912340341e-009 true resid norm 3.573383237647e-003 ||Ae||/||Ax|| 8.456355778617e-002 7 SNES Function norm 3.897321226706e-002 0 KSP Residual norm 4.766590050155e-002 0 KSP preconditioned resid norm 4.766590050155e-002 true resid norm 3.897321226706e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.528025220977e-009 1 KSP preconditioned resid norm 3.528025220977e-009 true resid norm 1.960313404416e-002 ||Ae||/||Ax|| 5.029899488355e-001 8 SNES Function norm 3.589720610234e-002 0 KSP Residual norm 4.289930864255e-002 0 KSP preconditioned resid norm 4.289930864255e-002 true resid norm 3.589720610234e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 4.133432759880e-009 1 KSP preconditioned resid norm 4.133432759880e-009 true resid norm 1.940005023620e-002 ||Ae||/||Ax|| 5.404334304149e-001 9 SNES Function norm 3.301230929196e-002 0 KSP Residual norm 3.860937624684e-002 0 KSP preconditioned resid norm 3.860937624684e-002 true resid norm 3.301230929196e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.211304264406e-010 1 KSP preconditioned resid norm 8.211304264406e-010 true resid norm 7.328958637952e-003 ||Ae||/||Ax|| 2.220068451781e-001 10 SNES Function norm 3.030900587812e-002 0 KSP Residual norm 3.474843739750e-002 0 KSP preconditioned resid norm 3.474843739750e-002 true resid norm 3.030900587812e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.265840938498e-009 1 KSP preconditioned resid norm 3.265840938498e-009 true resid norm 1.358870142093e-002 ||Ae||/||Ax|| 4.483387371916e-001 11 SNES Function norm 2.718409957830e-002 0 KSP Residual norm 3.013614572851e-002 0 KSP preconditioned resid norm 3.013614572851e-002 true resid norm 2.718409957830e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.652744245097e-009 1 KSP preconditioned resid norm 1.652744245097e-009 true resid norm 1.202105117910e-002 ||Ae||/||Ax|| 4.422089149754e-001 12 SNES Function norm 2.437348813670e-002 0 KSP Residual norm 2.629377332723e-002 0 KSP preconditioned resid norm 2.629377332723e-002 true resid norm 2.437348813670e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 3.959255153953e-009 1 KSP preconditioned resid norm 3.959255153953e-009 true resid norm 2.578989386428e-002 ||Ae||/||Ax|| 1.058112557367e+000 13 SNES Function norm 2.199784341474e-002 0 KSP Residual norm 2.328182605620e-002 0 KSP preconditioned resid norm 2.328182605620e-002 true resid norm 2.199784341474e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 8.915626892590e-010 1 KSP preconditioned resid norm 8.915626892590e-010 true resid norm 6.263373898192e-003 ||Ae||/||Ax|| 2.847267243477e-001 14 SNES Function norm 1.833996742657e-002 0 KSP Residual norm 1.794186796967e-002 0 KSP preconditioned resid norm 1.794186796967e-002 true resid norm 1.833996742657e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 1.694980794482e-009 1 KSP preconditioned resid norm 1.694980794482e-009 true resid norm 2.372076451714e-003 ||Ae||/||Ax|| 1.293391856453e-001 15 SNES Function norm 1.384131369872e-002 0 KSP Residual norm 1.074320273869e-002 0 KSP preconditioned resid norm 1.074320273869e-002 true resid norm 1.384131369872e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 7.118408455169e-010 1 KSP preconditioned resid norm 7.118408455169e-010 true resid norm 4.592913592019e-003 ||Ae||/||Ax|| 3.318264213927e-001 16 SNES Function norm 8.126551489387e-003 0 KSP Residual norm 2.817712852237e-009 0 KSP preconditioned resid norm 2.817712852237e-009 true resid norm 8.126551489387e-003 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 2.239560797627e-016 1 KSP preconditioned resid norm 2.239560797627e-016 true resid norm 1.053395037920e-009 ||Ae||/||Ax|| 1.296238680448e-007 17 SNES Function norm 4.871218193429e-008 0 KSP Residual norm 1.123675925695e-014 0 KSP preconditioned resid norm 1.123675925695e-014 true resid norm 4.871218193429e-008 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP Residual norm 5.307928330547e-022 1 KSP preconditioned resid norm 5.307928330547e-022 true resid norm 2.893354456541e-015 ||Ae||/||Ax|| 5.939693813026e-008 18 SNES Function norm 5.035279029783e-008 STEP 2 (Newton iterations: 18) diverged reason: -6 Rong > ----- Original Message ----- > From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 11:05 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > Ok, the linear solver looks like it is working ok. The likely problem is > that Jacobian does not match the function evaluation. > > Run the same thing but with the additional option -snes_mf_operator > > Then run with -snes_type test (instead of -snes_mf_operator). > > Barry > > On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: > >> Dear Barry, here is the output using -pc_type >> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >> the attached is the same and for clear reference. Thanks again for helps. >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> 0 SNES Function norm 7.071067811865e-002 >> 0 KSP Residual norm 9.965778978387e-002 >> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.823187455811e-017 >> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >> 1 SNES Function norm 6.401926523423e-002 >> 0 KSP Residual norm 8.969200212486e-002 >> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.106757475780e-016 >> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >> 2 SNES Function norm 5.849992149767e-002 >> 0 KSP Residual norm 8.072279488157e-002 >> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.268750073799e-017 >> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >> 3 SNES Function norm 5.376618503592e-002 >> 0 KSP Residual norm 7.265050969883e-002 >> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.677655733356e-017 >> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >> 4 SNES Function norm 4.956894354459e-002 >> 0 KSP Residual norm 6.538545411661e-002 >> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.557004153175e-017 >> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >> 5 SNES Function norm 4.575418613137e-002 >> 0 KSP Residual norm 5.884690496914e-002 >> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.470969262115e-017 >> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >> 6 SNES Function norm 4.223022245585e-002 >> 0 KSP Residual norm 5.296221144636e-002 >> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.255198782390e-017 >> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >> 7 SNES Function norm 3.894430065910e-002 >> 0 KSP Residual norm 4.766598785088e-002 >> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.322615478395e-017 >> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >> 8 SNES Function norm 3.586683371135e-002 >> 0 KSP Residual norm 4.289938708067e-002 >> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.181358328498e-017 >> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >> 9 SNES Function norm 3.298130202025e-002 >> 0 KSP Residual norm 3.860944676473e-002 >> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.635174776374e-017 >> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >> 10 SNES Function norm 3.027806208930e-002 >> 0 KSP Residual norm 3.474850078591e-002 >> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.389914053685e-017 >> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >> 11 SNES Function norm 2.749422924729e-002 >> 0 KSP Residual norm 3.081350823297e-002 >> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.640567497647e-017 >> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >> 12 SNES Function norm 2.437488247885e-002 >> 0 KSP Residual norm 2.633007441879e-002 >> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.772331460094e-017 >> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >> 13 SNES Function norm 2.079664278637e-002 >> 0 KSP Residual norm 2.104738289397e-002 >> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.650632708670e-017 >> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >> 14 SNES Function norm 1.657344626858e-002 >> 0 KSP Residual norm 1.454141853505e-002 >> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.129401160070e-017 >> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >> 15 SNES Function norm 1.484243752612e-002 >> 0 KSP Residual norm 5.241948491751e-009 >> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.729506849025e-024 >> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >> 16 SNES Function norm 2.828002157497e-008 >> 0 KSP Residual norm 6.042518362322e-015 >> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.272441346127e-030 >> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >> 17 SNES Function norm 2.960967020289e-008 >> STEP 0 (Newton iterations: 17) >> >> diverged reason: -6 >> >> >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Wednesday, May 11, 2011 2:54 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >> and >> send the outputs >> >> >> Barry >> >> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >> >>> by the way, the increment size is like that >>> for a 100 lengh model, the increment is set to 0.05, >>> the engineering strain is around 5%% per load step. >>> This is already too small increment size for a large deformation >>> analysis. >>> a 0.5 increment size leads to both linear search and trust region >>> failed. >>> linear search failed for 0.05 while trust region converges with 17 >>> Newton >>> iterations each load step. >>> Rong >>> >>> ----- Original Message ----- From: "Tian(ICT)" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 11:37 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>>> First, thanks again, the issue was gone. >>>> >>>> I just followed up with some test results. >>>> I have tested SNES using one finite element for a geometric large >>>> deformation problem. >>>> Those are just the very early test results so they may be not telling >>>> what happened exactly. >>>> For the displacement controlled load, I found that convergence is much >>>> slower than that of force loading. >>>> Even worse, linear search is so sensitive to the displacement increment >>>> and diverged no matter what the increment size was used (too small >>>> incremnt also led to diverged soloution (-6 reason), trust region works >>>> well in the sense of not sensitive to the displacement increment, but >>>> during each load step, it requires around ten to several tens of Newton >>>> interations whereas for the force loading case and the almost same >>>> amount of deformation, this is normally 3. This is against my >>>> expectation. Any hint? >>>> >>>> Rong >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>> >>>>> Dear Barry, Thanks a lot for quick answering. >>>>> I checked the development documents and found the new version of >>>>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>>>> >>>>> I followed up with more details. >>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>>>> solvers. >>>>> I used a displacement-controlled load (as this type of loading works >>>>> well for all cases). >>>>> This is the reason the nonzero prescribed boundary came up. >>>>> >>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>> nonzero prescribed boundary. >>>>> In FormFunction, I modified the solution to the known solution(this >>>>> should not be necessary as the modified Jacobian and rhs should give >>>>> the prescribed solution also) >>>> >>>> You should not do it this way. See below. >>>>> >>>>> Now I found another issue, no matter if I prescried the solution or >>>>> not >>>>> in FormFunction, >>>>> SNES solver always call FormFunction and never call FormJacobian. >>>> >>>> The only reason it would not call FormJacobian is if decided that the >>>> residual norm was small enough before any Newton steps; for example if >>>> the FormFunction() computed exactly the zero function initially. When >>>> you run with -snes_monitor -ksp_monitor what does it print for residual >>>> norms. >>>> >>>>> Of course the solver finally diverged or converged to a zero solution. >>>>> >>>>> So my quick follow up question is How a displacement-controled load is >>>>> done corrently in Petsc 2.3.3? >>>> >>>> To do it in 2.3.3 simply have for those components of F() the formula >>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() >>>> for >>>> those rows >>>> >>>> We strongly urge you to upgrade to the latest PETSc before doing >>>> anything further. >>>> >>>> >>>> Barry >>>> >>>>> >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Barry Smith" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>> >>>>> In petsc-dev >>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>> modified the calling sequence for MatZeroRows() so that it can >>>>> automatically adjust the appropriate right hand side values for the >>>>> zeroed rows to support zero or non-zero prescribed boundary conditions >>>>> easily. >>>>> >>>>> Barry >>>>> >>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>> >>>>>> Dear all, >>>>>> >>>>>> I got this question long ago and searched the prior posting but did >>>>>> not find the solution. >>>>>> The question is about nonzero prescribed boundary condition. >>>>>> My understanding is that MatZeroRows() works only for zero prescribed >>>>>> value, not non-zero value. >>>>>> For the non-zero values, we have to remove the rows associated with >>>>>> the boundary, but this >>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>> also be removed. >>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>> prescribed boundary and if so how to do it simply? >>>>>> >>>>>> Rong >>>>> >>>>> >>>> >>>> >>> >> >> > > > From bsmith at mcs.anl.gov Wed May 11 08:02:27 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 11 May 2011 08:02:27 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <1302D684D3644298A3F2528BDEE08857@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop> <1302D684D3644298A3F2528BDEE08857@rti458laptop> Message-ID: >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 9.05702e-008 difference 3.94971 >> Norm of matrix ratio 3.6015e-007 difference 15.7059 >> Norm of matrix ratio 3.73941e-007 difference 16.3073 Based on this it appears the Jacobian at the initial point is correct but 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 this indicates that the linear solver is not actually solving the linear system. One would expect a large drop in the "true resid norm" and one does not see that. This could be caused by 1) the computed Jacobian is wrong or 2) the Jacobian is singular and that is not being taken into account in the linear solver. Please rerun with the options -pc_type lu -ksp_monitor_true_residual -snes_monitor BUT NOT THE OPTION -snes_mf_operator and send the same output. Barry On May 11, 2011, at 1:46 AM, Tian(ICT) wrote: > Pls ignore my previous email. > > The followings are the test results: > the previous error in using -snes_mf_opertor was due to my own code. > After modification, i.e. using different A and B and assemble both > and in FormJacobian the native jacobian was assembled into B. > Then with -snes_mf_operator led to the results shown below. > The first three steps converged but with a diverged reason -6 at step 4. > > > With additional -snes_type test, the results were > >> "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 9.05702e-008 difference 3.94971 >> Norm of matrix ratio 3.6015e-007 difference 15.7059 >> Norm of matrix ratio 3.73941e-007 difference 16.3073 >> [0]PETSC ERROR: SNESSolve() line 1871 in >> src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c >> [0]PETSC ERROR: User provided function() line 572 in >> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp >> [0]PETSC ERROR: User provided function() line 40 in >> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp >> >> >> "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP Residual norm 9.965778978387e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.102368266281e-009 > 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 > 1 SNES Function norm 6.401926196865e-002 > 0 KSP Residual norm 8.969200166800e-002 > 0 KSP preconditioned resid norm 8.969200166800e-002 true resid norm 6.401926196865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.492849351675e-009 > 1 KSP preconditioned resid norm 8.492849351675e-009 true resid norm 1.400035213572e-002 ||Ae||/||Ax|| 2.186896834670e-001 > 2 SNES Function norm 5.849991037388e-002 > 0 KSP Residual norm 8.072279482667e-002 > 0 KSP preconditioned resid norm 8.072279482667e-002 true resid norm 5.849991037388e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.610510067555e-009 > 1 KSP preconditioned resid norm 3.610510067555e-009 true resid norm 4.757549946973e-003 ||Ae||/||Ax|| 8.132576471599e-002 > 3 SNES Function norm 5.376617142827e-002 > 0 KSP Residual norm 7.265050986128e-002 > 0 KSP preconditioned resid norm 7.265050986128e-002 true resid norm 5.376617142827e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.941165647221e-009 > 1 KSP preconditioned resid norm 6.941165647221e-009 true resid norm 3.935270526562e-002 ||Ae||/||Ax|| 7.319231446138e-001 > 4 SNES Function norm 4.956888929193e-002 > 0 KSP Residual norm 6.538545408372e-002 > 0 KSP preconditioned resid norm 6.538545408372e-002 true resid norm 4.956888929193e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.457970649220e-009 > 1 KSP preconditioned resid norm 3.457970649220e-009 true resid norm 2.037700110050e-002 ||Ae||/||Ax|| 4.110844804388e-001 > 5 SNES Function norm 4.575415001780e-002 > 0 KSP Residual norm 5.884690527647e-002 > 0 KSP preconditioned resid norm 5.884690527647e-002 true resid norm 4.575415001780e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.832662187541e-009 > 1 KSP preconditioned resid norm 2.832662187541e-009 true resid norm 1.662420124938e-002 ||Ae||/||Ax|| 3.633375604818e-001 > 6 SNES Function norm 4.223018337199e-002 > 0 KSP Residual norm 5.296221162333e-002 > 0 KSP preconditioned resid norm 5.296221162333e-002 true resid norm 4.223018337199e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.091667005099e-009 > 1 KSP preconditioned resid norm 3.091667005099e-009 true resid norm 2.235399010718e-002 ||Ae||/||Ax|| 5.293367994705e-001 > 7 SNES Function norm 3.894426990170e-002 > 0 KSP Residual norm 4.766598828506e-002 > 0 KSP preconditioned resid norm 4.766598828506e-002 true resid norm 3.894426990170e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.223811830585e-009 > 1 KSP preconditioned resid norm 4.223811830585e-009 true resid norm 2.831156780040e-002 ||Ae||/||Ax|| 7.269764684732e-001 > 8 SNES Function norm 3.586679674893e-002 > 0 KSP Residual norm 4.289938722959e-002 > 0 KSP preconditioned resid norm 4.289938722959e-002 true resid norm 3.586679674893e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.504535460429e-009 > 1 KSP preconditioned resid norm 3.504535460429e-009 true resid norm 2.254598873838e-002 ||Ae||/||Ax|| 6.286033541328e-001 > 9 SNES Function norm 3.298126871313e-002 > 0 KSP Residual norm 3.860944657267e-002 > 0 KSP preconditioned resid norm 3.860944657267e-002 true resid norm 3.298126871313e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.439084585574e-009 > 1 KSP preconditioned resid norm 4.439084585574e-009 true resid norm 6.936792391193e-003 ||Ae||/||Ax|| 2.103252137305e-001 > 10 SNES Function norm 3.027806037514e-002 > 0 KSP Residual norm 3.474850031738e-002 > 0 KSP preconditioned resid norm 3.474850031738e-002 true resid norm 3.027806037514e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.476854671229e-009 > 1 KSP preconditioned resid norm 3.476854671229e-009 true resid norm 1.935717696630e-002 ||Ae||/||Ax|| 6.393136391984e-001 > 11 SNES Function norm 2.775064646763e-002 > 0 KSP Residual norm 3.127364888698e-002 > 0 KSP preconditioned resid norm 3.127364888698e-002 true resid norm 2.775064646763e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.380857715198e-009 > 1 KSP preconditioned resid norm 2.380857715198e-009 true resid norm 1.646856223644e-002 ||Ae||/||Ax|| 5.934478771746e-001 > 12 SNES Function norm 2.430399371900e-002 > 0 KSP Residual norm 2.602654480862e-002 > 0 KSP preconditioned resid norm 2.602654480862e-002 true resid norm 2.430399371900e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.556142726483e-009 > 1 KSP preconditioned resid norm 2.556142726483e-009 true resid norm 1.794651743617e-002 ||Ae||/||Ax|| 7.384184526900e-001 > 13 SNES Function norm 2.029256664631e-002 > 0 KSP Residual norm 1.954645630691e-002 > 0 KSP preconditioned resid norm 1.954645630691e-002 true resid norm 2.029256664631e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.913585412092e-010 > 1 KSP preconditioned resid norm 3.913585412092e-010 true resid norm 1.903675728342e-003 ||Ae||/||Ax|| 9.381148089946e-002 > 14 SNES Function norm 1.566638821565e-002 > 0 KSP Residual norm 1.231404128086e-002 > 0 KSP preconditioned resid norm 1.231404128086e-002 true resid norm 1.566638821565e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.258598971379e-010 > 1 KSP preconditioned resid norm 7.258598971379e-010 true resid norm 1.221317129610e-003 ||Ae||/||Ax|| 7.795779810881e-002 > 15 SNES Function norm 1.064422587384e-002 > 0 KSP Residual norm 3.723145759360e-009 > 0 KSP preconditioned resid norm 3.723145759360e-009 true resid norm 1.064422587384e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.546825086207e-016 > 1 KSP preconditioned resid norm 5.546825086207e-016 true resid norm 2.984647735625e-009 ||Ae||/||Ax|| 2.804006389003e-007 > 16 SNES Function norm 9.689226075617e-008 > 0 KSP Residual norm 1.643080767133e-014 > 0 KSP preconditioned resid norm 1.643080767133e-014 true resid norm 9.689226075617e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.291965935374e-022 > 1 KSP preconditioned resid norm 7.291965935374e-022 true resid norm 5.001850432973e-015 ||Ae||/||Ax|| 5.162280654758e-008 > 17 SNES Function norm 3.747290584108e-008 > STEP 0 (Newton iterations: 17) > > 0 SNES Function norm 7.071067811866e-002 > 0 KSP Residual norm 9.965769844003e-002 > 0 KSP preconditioned resid norm 9.965769844003e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.919247724727e-009 > 1 KSP preconditioned resid norm 4.919247724727e-009 true resid norm 2.073848008334e-002 ||Ae||/||Ax|| 2.932863979685e-001 > 1 SNES Function norm 6.402041310994e-002 > 0 KSP Residual norm 8.969192038029e-002 > 0 KSP preconditioned resid norm 8.969192038029e-002 true resid norm 6.402041310994e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.072962844649e-009 > 1 KSP preconditioned resid norm 4.072962844649e-009 true resid norm 2.116458841832e-002 ||Ae||/||Ax|| 3.305912503559e-001 > 2 SNES Function norm 5.850358277995e-002 > 0 KSP Residual norm 8.072272129834e-002 > 0 KSP preconditioned resid norm 8.072272129834e-002 true resid norm 5.850358277995e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.535693836801e-009 > 1 KSP preconditioned resid norm 1.535693836801e-009 true resid norm 1.378413836547e-002 ||Ae||/||Ax|| 2.356118670085e-001 > 3 SNES Function norm 5.377276379928e-002 > 0 KSP Residual norm 7.265044334812e-002 > 0 KSP preconditioned resid norm 7.265044334812e-002 true resid norm 5.377276379928e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.196914267894e-009 > 1 KSP preconditioned resid norm 8.196914267894e-009 true resid norm 1.643681592299e-002 ||Ae||/||Ax|| 3.056717706448e-001 > 4 SNES Function norm 4.957825627064e-002 > 0 KSP Residual norm 6.538539498475e-002 > 0 KSP preconditioned resid norm 6.538539498475e-002 true resid norm 4.957825627064e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.605278143372e-009 > 1 KSP preconditioned resid norm 1.605278143372e-009 true resid norm 1.365749144340e-002 ||Ae||/||Ax|| 2.754734125550e-001 > 5 SNES Function norm 4.576573391352e-002 > 0 KSP Residual norm 5.884685187863e-002 > 0 KSP preconditioned resid norm 5.884685187863e-002 true resid norm 4.576573391352e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.917790130377e-009 > 1 KSP preconditioned resid norm 3.917790130377e-009 true resid norm 2.918137698804e-002 ||Ae||/||Ax|| 6.376250196967e-001 > 6 SNES Function norm 4.224348383506e-002 > 0 KSP Residual norm 5.296216341013e-002 > 0 KSP preconditioned resid norm 5.296216341013e-002 true resid norm 4.224348383506e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.384279752582e-009 > 1 KSP preconditioned resid norm 1.384279752582e-009 true resid norm 8.645143617596e-003 ||Ae||/||Ax|| 2.046503468168e-001 > 7 SNES Function norm 3.895875122953e-002 > 0 KSP Residual norm 4.766594475257e-002 > 0 KSP preconditioned resid norm 4.766594475257e-002 true resid norm 3.895875122953e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.442999105246e-009 > 1 KSP preconditioned resid norm 5.442999105246e-009 true resid norm 3.199366262059e-002 ||Ae||/||Ax|| 8.212188946227e-001 > 8 SNES Function norm 3.588203219083e-002 > 0 KSP Residual norm 4.289934859308e-002 > 0 KSP preconditioned resid norm 4.289934859308e-002 true resid norm 3.588203219083e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.069272581944e-009 > 1 KSP preconditioned resid norm 5.069272581944e-009 true resid norm 3.315026802067e-002 ||Ae||/||Ax|| 9.238681868510e-001 > 9 SNES Function norm 3.299674865285e-002 > 0 KSP Residual norm 3.860941169595e-002 > 0 KSP preconditioned resid norm 3.860941169595e-002 true resid norm 3.299674865285e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.314510318621e-009 > 1 KSP preconditioned resid norm 2.314510318621e-009 true resid norm 1.422355468145e-002 ||Ae||/||Ax|| 4.310592789335e-001 > 10 SNES Function norm 3.020092421335e-002 > 0 KSP Residual norm 3.457428537321e-002 > 0 KSP preconditioned resid norm 3.457428537321e-002 true resid norm 3.020092421335e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.171225245965e-009 > 1 KSP preconditioned resid norm 3.171225245965e-009 true resid norm 6.700877909403e-003 ||Ae||/||Ax|| 2.218765843742e-001 > 11 SNES Function norm 2.761432326644e-002 > 0 KSP Residual norm 3.100402216444e-002 > 0 KSP preconditioned resid norm 3.100402216444e-002 true resid norm 2.761432326644e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.317753799744e-009 > 1 KSP preconditioned resid norm 2.317753799744e-009 true resid norm 1.143024510909e-002 ||Ae||/||Ax|| 4.139245057287e-001 > 12 SNES Function norm 2.483228477543e-002 > 0 KSP Residual norm 2.716039584199e-002 > 0 KSP preconditioned resid norm 2.716039584199e-002 true resid norm 2.483228477543e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.024860175617e-009 > 1 KSP preconditioned resid norm 1.024860175617e-009 true resid norm 5.071961436716e-003 ||Ae||/||Ax|| 2.042486820115e-001 > 13 SNES Function norm 2.155599009420e-002 > 0 KSP Residual norm 2.247739022940e-002 > 0 KSP preconditioned resid norm 2.247739022940e-002 true resid norm 2.155599009420e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.231896568498e-009 > 1 KSP preconditioned resid norm 1.231896568498e-009 true resid norm 2.253717888475e-003 ||Ae||/||Ax|| 1.045518150002e-001 > 14 SNES Function norm 1.758159262430e-002 > 0 KSP Residual norm 1.640654408633e-002 > 0 KSP preconditioned resid norm 1.640654408633e-002 true resid norm 1.758159262430e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.203409491551e-009 > 1 KSP preconditioned resid norm 1.203409491551e-009 true resid norm 3.340286893989e-003 ||Ae||/||Ax|| 1.899877312236e-001 > 15 SNES Function norm 1.275874713861e-002 > 0 KSP Residual norm 8.737890404296e-003 > 0 KSP preconditioned resid norm 8.737890404296e-003 true resid norm 1.275874713861e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.862616851249e-010 > 1 KSP preconditioned resid norm 3.862616851249e-010 true resid norm 9.158071400861e-004 ||Ae||/||Ax|| 7.177876715771e-002 > 16 SNES Function norm 5.367994969686e-003 > 0 KSP Residual norm 1.873266786098e-009 > 0 KSP preconditioned resid norm 1.873266786098e-009 true resid norm 5.367994969686e-003 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.236612780542e-017 > 1 KSP preconditioned resid norm 9.236612780542e-017 true resid norm 4.487656305545e-010 ||Ae||/||Ax|| 8.360023306445e-008 > 17 SNES Function norm 1.843858578272e-008 > 0 KSP Residual norm 3.418191528615e-015 > 0 KSP preconditioned resid norm 3.418191528615e-015 true resid norm 1.843858578272e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.091028447697e-022 > 1 KSP preconditioned resid norm 2.091028447697e-022 true resid norm 9.356082852780e-016 ||Ae||/||Ax|| 5.074186796662e-008 > 18 SNES Function norm 1.582175998776e-008 > STEP 1 (Newton iterations: 18) > > 0 SNES Function norm 7.071067811866e-002 > 0 KSP Residual norm 9.965760716669e-002 > 0 KSP preconditioned resid norm 9.965760716669e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.158690871520e-009 > 1 KSP preconditioned resid norm 4.158690871520e-009 true resid norm 1.356168089543e-002 ||Ae||/||Ax|| 1.917911305090e-001 > 1 SNES Function norm 6.402155946870e-002 > 0 KSP Residual norm 8.969183775926e-002 > 0 KSP preconditioned resid norm 8.969183775926e-002 true resid norm 6.402155946870e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.922035131300e-009 > 1 KSP preconditioned resid norm 7.922035131300e-009 true resid norm 4.173737132742e-002 ||Ae||/||Ax|| 6.519268145573e-001 > 2 SNES Function norm 5.850726192206e-002 > 0 KSP Residual norm 8.072264774465e-002 > 0 KSP preconditioned resid norm 8.072264774465e-002 true resid norm 5.850726192206e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.356474211542e-009 > 1 KSP preconditioned resid norm 4.356474211542e-009 true resid norm 2.083500604436e-002 ||Ae||/||Ax|| 3.561097436437e-001 > 3 SNES Function norm 5.377933295280e-002 > 0 KSP Residual norm 7.265037686612e-002 > 0 KSP preconditioned resid norm 7.265037686612e-002 true resid norm 5.377933295280e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.772318530630e-009 > 1 KSP preconditioned resid norm 8.772318530630e-009 true resid norm 5.284958913103e-002 ||Ae||/||Ax|| 9.827118751623e-001 > 4 SNES Function norm 4.958751191600e-002 > 0 KSP Residual norm 6.538533386375e-002 > 0 KSP preconditioned resid norm 6.538533386375e-002 true resid norm 4.958751191600e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.128741903850e-009 > 1 KSP preconditioned resid norm 1.128741903850e-009 true resid norm 2.526410501148e-003 ||Ae||/||Ax|| 5.094852319729e-002 > 5 SNES Function norm 4.577729047509e-002 > 0 KSP Residual norm 5.884679667348e-002 > 0 KSP preconditioned resid norm 5.884679667348e-002 true resid norm 4.577729047509e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.763627011382e-009 > 1 KSP preconditioned resid norm 5.763627011382e-009 true resid norm 1.051243252216e-002 ||Ae||/||Ax|| 2.296429608013e-001 > 6 SNES Function norm 4.225677503639e-002 > 0 KSP Residual norm 5.296211439112e-002 > 0 KSP preconditioned resid norm 5.296211439112e-002 true resid norm 4.225677503639e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.022912340341e-009 > 1 KSP preconditioned resid norm 1.022912340341e-009 true resid norm 3.573383237647e-003 ||Ae||/||Ax|| 8.456355778617e-002 > 7 SNES Function norm 3.897321226706e-002 > 0 KSP Residual norm 4.766590050155e-002 > 0 KSP preconditioned resid norm 4.766590050155e-002 true resid norm 3.897321226706e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.528025220977e-009 > 1 KSP preconditioned resid norm 3.528025220977e-009 true resid norm 1.960313404416e-002 ||Ae||/||Ax|| 5.029899488355e-001 > 8 SNES Function norm 3.589720610234e-002 > 0 KSP Residual norm 4.289930864255e-002 > 0 KSP preconditioned resid norm 4.289930864255e-002 true resid norm 3.589720610234e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.133432759880e-009 > 1 KSP preconditioned resid norm 4.133432759880e-009 true resid norm 1.940005023620e-002 ||Ae||/||Ax|| 5.404334304149e-001 > 9 SNES Function norm 3.301230929196e-002 > 0 KSP Residual norm 3.860937624684e-002 > 0 KSP preconditioned resid norm 3.860937624684e-002 true resid norm 3.301230929196e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.211304264406e-010 > 1 KSP preconditioned resid norm 8.211304264406e-010 true resid norm 7.328958637952e-003 ||Ae||/||Ax|| 2.220068451781e-001 > 10 SNES Function norm 3.030900587812e-002 > 0 KSP Residual norm 3.474843739750e-002 > 0 KSP preconditioned resid norm 3.474843739750e-002 true resid norm 3.030900587812e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.265840938498e-009 > 1 KSP preconditioned resid norm 3.265840938498e-009 true resid norm 1.358870142093e-002 ||Ae||/||Ax|| 4.483387371916e-001 > 11 SNES Function norm 2.718409957830e-002 > 0 KSP Residual norm 3.013614572851e-002 > 0 KSP preconditioned resid norm 3.013614572851e-002 true resid norm 2.718409957830e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.652744245097e-009 > 1 KSP preconditioned resid norm 1.652744245097e-009 true resid norm 1.202105117910e-002 ||Ae||/||Ax|| 4.422089149754e-001 > 12 SNES Function norm 2.437348813670e-002 > 0 KSP Residual norm 2.629377332723e-002 > 0 KSP preconditioned resid norm 2.629377332723e-002 true resid norm 2.437348813670e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.959255153953e-009 > 1 KSP preconditioned resid norm 3.959255153953e-009 true resid norm 2.578989386428e-002 ||Ae||/||Ax|| 1.058112557367e+000 > 13 SNES Function norm 2.199784341474e-002 > 0 KSP Residual norm 2.328182605620e-002 > 0 KSP preconditioned resid norm 2.328182605620e-002 true resid norm 2.199784341474e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.915626892590e-010 > 1 KSP preconditioned resid norm 8.915626892590e-010 true resid norm 6.263373898192e-003 ||Ae||/||Ax|| 2.847267243477e-001 > 14 SNES Function norm 1.833996742657e-002 > 0 KSP Residual norm 1.794186796967e-002 > 0 KSP preconditioned resid norm 1.794186796967e-002 true resid norm 1.833996742657e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.694980794482e-009 > 1 KSP preconditioned resid norm 1.694980794482e-009 true resid norm 2.372076451714e-003 ||Ae||/||Ax|| 1.293391856453e-001 > 15 SNES Function norm 1.384131369872e-002 > 0 KSP Residual norm 1.074320273869e-002 > 0 KSP preconditioned resid norm 1.074320273869e-002 true resid norm 1.384131369872e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.118408455169e-010 > 1 KSP preconditioned resid norm 7.118408455169e-010 true resid norm 4.592913592019e-003 ||Ae||/||Ax|| 3.318264213927e-001 > 16 SNES Function norm 8.126551489387e-003 > 0 KSP Residual norm 2.817712852237e-009 > 0 KSP preconditioned resid norm 2.817712852237e-009 true resid norm 8.126551489387e-003 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.239560797627e-016 > 1 KSP preconditioned resid norm 2.239560797627e-016 true resid norm 1.053395037920e-009 ||Ae||/||Ax|| 1.296238680448e-007 > 17 SNES Function norm 4.871218193429e-008 > 0 KSP Residual norm 1.123675925695e-014 > 0 KSP preconditioned resid norm 1.123675925695e-014 true resid norm 4.871218193429e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.307928330547e-022 > 1 KSP preconditioned resid norm 5.307928330547e-022 true resid norm 2.893354456541e-015 ||Ae||/||Ax|| 5.939693813026e-008 > 18 SNES Function norm 5.035279029783e-008 > STEP 2 (Newton iterations: 18) > > diverged reason: -6 > > > Rong > >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Wednesday, May 11, 2011 11:05 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> Ok, the linear solver looks like it is working ok. The likely problem is >> that Jacobian does not match the function evaluation. >> >> Run the same thing but with the additional option -snes_mf_operator >> >> Then run with -snes_type test (instead of -snes_mf_operator). >> >> Barry >> >> On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: >> >>> Dear Barry, here is the output using -pc_type >>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>> the attached is the same and for clear reference. Thanks again for helps. >>> >>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>> 0 SNES Function norm 7.071067811865e-002 >>> 0 KSP Residual norm 9.965778978387e-002 >>> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >>> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.823187455811e-017 >>> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >>> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >>> 1 SNES Function norm 6.401926523423e-002 >>> 0 KSP Residual norm 8.969200212486e-002 >>> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >>> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.106757475780e-016 >>> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >>> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >>> 2 SNES Function norm 5.849992149767e-002 >>> 0 KSP Residual norm 8.072279488157e-002 >>> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >>> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.268750073799e-017 >>> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >>> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >>> 3 SNES Function norm 5.376618503592e-002 >>> 0 KSP Residual norm 7.265050969883e-002 >>> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >>> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.677655733356e-017 >>> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >>> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >>> 4 SNES Function norm 4.956894354459e-002 >>> 0 KSP Residual norm 6.538545411661e-002 >>> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >>> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 9.557004153175e-017 >>> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >>> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >>> 5 SNES Function norm 4.575418613137e-002 >>> 0 KSP Residual norm 5.884690496914e-002 >>> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >>> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 5.470969262115e-017 >>> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >>> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >>> 6 SNES Function norm 4.223022245585e-002 >>> 0 KSP Residual norm 5.296221144636e-002 >>> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >>> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 8.255198782390e-017 >>> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >>> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >>> 7 SNES Function norm 3.894430065910e-002 >>> 0 KSP Residual norm 4.766598785088e-002 >>> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >>> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 3.322615478395e-017 >>> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >>> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >>> 8 SNES Function norm 3.586683371135e-002 >>> 0 KSP Residual norm 4.289938708067e-002 >>> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >>> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.181358328498e-017 >>> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >>> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >>> 9 SNES Function norm 3.298130202025e-002 >>> 0 KSP Residual norm 3.860944676473e-002 >>> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >>> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 4.635174776374e-017 >>> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >>> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >>> 10 SNES Function norm 3.027806208930e-002 >>> 0 KSP Residual norm 3.474850078591e-002 >>> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >>> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.389914053685e-017 >>> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >>> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >>> 11 SNES Function norm 2.749422924729e-002 >>> 0 KSP Residual norm 3.081350823297e-002 >>> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >>> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.640567497647e-017 >>> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >>> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >>> 12 SNES Function norm 2.437488247885e-002 >>> 0 KSP Residual norm 2.633007441879e-002 >>> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >>> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.772331460094e-017 >>> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >>> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >>> 13 SNES Function norm 2.079664278637e-002 >>> 0 KSP Residual norm 2.104738289397e-002 >>> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >>> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.650632708670e-017 >>> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >>> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >>> 14 SNES Function norm 1.657344626858e-002 >>> 0 KSP Residual norm 1.454141853505e-002 >>> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >>> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.129401160070e-017 >>> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >>> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >>> 15 SNES Function norm 1.484243752612e-002 >>> 0 KSP Residual norm 5.241948491751e-009 >>> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >>> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.729506849025e-024 >>> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >>> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >>> 16 SNES Function norm 2.828002157497e-008 >>> 0 KSP Residual norm 6.042518362322e-015 >>> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >>> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.272441346127e-030 >>> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >>> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >>> 17 SNES Function norm 2.960967020289e-008 >>> STEP 0 (Newton iterations: 17) >>> >>> diverged reason: -6 >>> >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Wednesday, May 11, 2011 2:54 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and >>> send the outputs >>> >>> >>> Barry >>> >>> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >>> >>>> by the way, the increment size is like that >>>> for a 100 lengh model, the increment is set to 0.05, >>>> the engineering strain is around 5%% per load step. >>>> This is already too small increment size for a large deformation >>>> analysis. >>>> a 0.5 increment size leads to both linear search and trust region failed. >>>> linear search failed for 0.05 while trust region converges with 17 Newton >>>> iterations each load step. >>>> Rong >>>> >>>> ----- Original Message ----- From: "Tian(ICT)" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 11:37 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>>> First, thanks again, the issue was gone. >>>>> >>>>> I just followed up with some test results. >>>>> I have tested SNES using one finite element for a geometric large >>>>> deformation problem. >>>>> Those are just the very early test results so they may be not telling >>>>> what happened exactly. >>>>> For the displacement controlled load, I found that convergence is much >>>>> slower than that of force loading. >>>>> Even worse, linear search is so sensitive to the displacement increment >>>>> and diverged no matter what the increment size was used (too small >>>>> incremnt also led to diverged soloution (-6 reason), trust region works >>>>> well in the sense of not sensitive to the displacement increment, but >>>>> during each load step, it requires around ten to several tens of Newton >>>>> interations whereas for the force loading case and the almost same >>>>> amount of deformation, this is normally 3. This is against my >>>>> expectation. Any hint? >>>>> >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Barry Smith" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>> >>>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>>> >>>>>> Dear Barry, Thanks a lot for quick answering. >>>>>> I checked the development documents and found the new version of >>>>>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>>>>> >>>>>> I followed up with more details. >>>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>>>>> solvers. >>>>>> I used a displacement-controlled load (as this type of loading works >>>>>> well for all cases). >>>>>> This is the reason the nonzero prescribed boundary came up. >>>>>> >>>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>>> nonzero prescribed boundary. >>>>>> In FormFunction, I modified the solution to the known solution(this >>>>>> should not be necessary as the modified Jacobian and rhs should give >>>>>> the prescribed solution also) >>>>> >>>>> You should not do it this way. See below. >>>>>> >>>>>> Now I found another issue, no matter if I prescried the solution or not >>>>>> in FormFunction, >>>>>> SNES solver always call FormFunction and never call FormJacobian. >>>>> >>>>> The only reason it would not call FormJacobian is if decided that the >>>>> residual norm was small enough before any Newton steps; for example if >>>>> the FormFunction() computed exactly the zero function initially. When >>>>> you run with -snes_monitor -ksp_monitor what does it print for residual >>>>> norms. >>>>> >>>>>> Of course the solver finally diverged or converged to a zero solution. >>>>>> >>>>>> So my quick follow up question is How a displacement-controled load is >>>>>> done corrently in Petsc 2.3.3? >>>>> >>>>> To do it in 2.3.3 simply have for those components of F() the formula >>>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for >>>>> those rows >>>>> >>>>> We strongly urge you to upgrade to the latest PETSc before doing >>>>> anything further. >>>>> >>>>> >>>>> Barry >>>>> >>>>>> >>>>>> Rong >>>>>> >>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>> To: "PETSc users list" >>>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>> >>>>>> >>>>>> >>>>>> In petsc-dev >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>>> modified the calling sequence for MatZeroRows() so that it can >>>>>> automatically adjust the appropriate right hand side values for the >>>>>> zeroed rows to support zero or non-zero prescribed boundary conditions >>>>>> easily. >>>>>> >>>>>> Barry >>>>>> >>>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>>> >>>>>>> Dear all, >>>>>>> >>>>>>> I got this question long ago and searched the prior posting but did >>>>>>> not find the solution. >>>>>>> The question is about nonzero prescribed boundary condition. >>>>>>> My understanding is that MatZeroRows() works only for zero prescribed >>>>>>> value, not non-zero value. >>>>>>> For the non-zero values, we have to remove the rows associated with >>>>>>> the boundary, but this >>>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>>> also be removed. >>>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>>> prescribed boundary and if so how to do it simply? >>>>>>> >>>>>>> Rong >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> > From rongtian at ncic.ac.cn Wed May 11 08:44:21 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Wed, 11 May 2011 21:44:21 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop><1302D684D3644298A3F2528BDEE08857@rti458laptop> Message-ID: <32DB8456522B4FABB725933C113D66D1@rti458laptop> C:\cygwin\home\rabbit\pGFEM-test>pgfem -pc_type lu -ksp_monitor_true_residual -s nes_monitor atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067811865e-002 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.07106781 1865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm 8.84729888 5656e-011 ||Ae||/||Ax|| 1.251197007446e-009 1 SNES Function norm 6.401926523423e-002 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm 6.40192652 3423e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm 6.21183006 7439e-011 ||Ae||/||Ax|| 9.703063671087e-010 2 SNES Function norm 5.849992149767e-002 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm 5.84999214 9767e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm 3.80243103 6387e-011 ||Ae||/||Ax|| 6.499890835816e-010 3 SNES Function norm 5.376618503592e-002 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm 5.37661850 3592e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm 8.12039778 8686e-011 ||Ae||/||Ax|| 1.510316899602e-009 4 SNES Function norm 4.956894354459e-002 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm 4.95689435 4459e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm 2.94425080 2029e-011 ||Ae||/||Ax|| 5.939708598754e-010 5 SNES Function norm 4.575418613137e-002 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm 4.57541861 3137e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm 3.65900316 6095e-011 ||Ae||/||Ax|| 7.997089393284e-010 6 SNES Function norm 4.223022245585e-002 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm 4.22302224 5585e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm 1.95554565 8933e-011 ||Ae||/||Ax|| 4.630678090739e-010 7 SNES Function norm 3.894430065910e-002 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm 3.89443006 5910e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm 3.48532814 8673e-011 ||Ae||/||Ax|| 8.949520442496e-010 8 SNES Function norm 3.586683371135e-002 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm 3.58668337 1135e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm 3.24690281 8086e-011 ||Ae||/||Ax|| 9.052660862724e-010 9 SNES Function norm 3.298130202025e-002 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm 3.29813020 2025e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm 1.49751684 2272e-011 ||Ae||/||Ax|| 4.540502498513e-010 10 SNES Function norm 3.027806208930e-002 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm 3.02780620 8930e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm 6.00744088 8596e-011 ||Ae||/||Ax|| 1.984090286517e-009 11 SNES Function norm 2.749422924729e-002 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm 2.74942292 4729e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm 1.28163829 5853e-011 ||Ae||/||Ax|| 4.661481085089e-010 12 SNES Function norm 2.437488247885e-002 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm 2.43748824 7885e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm 1.91821249 6143e-011 ||Ae||/||Ax|| 7.869627670236e-010 13 SNES Function norm 2.079664278637e-002 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm 2.07966427 8637e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm 2.31637196 7362e-011 ||Ae||/||Ax|| 1.113820144509e-009 14 SNES Function norm 1.657344626858e-002 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm 1.65734462 6858e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm 7.88549932 7559e-012 ||Ae||/||Ax|| 4.757911661686e-010 15 SNES Function norm 1.484243752612e-002 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm 1.48424375 2612e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm 6.38667785 1085e-018 ||Ae||/||Ax|| 4.302984492839e-016 16 SNES Function norm 2.828002157497e-008 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm 2.82800215 7497e-008 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm 1.11285769 8032e-023 ||Ae||/||Ax|| 3.935137372797e-016 17 SNES Function norm 2.960967020289e-008 STEP 0 (Newton iterations: 17) diverged reason: -6 C:\cygwin\home\rabbit\pGFEM-test> ==================HOWEVER================= -snes_type tr seems work C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_type tr atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 STEP 0 (Newton iterations: 12) STEP 1 (Newton iterations: 11) STEP 2 (Newton iterations: 11) ... STEP 70 (Newton iterations: 14) ^C C:\cygwin\home\rabbit\pGFEM-test> ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Wednesday, May 11, 2011 9:02 PM Subject: Re: [petsc-users] nonzero prescribed boundary condition >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 9.05702e-008 difference 3.94971 >> Norm of matrix ratio 3.6015e-007 difference 15.7059 >> Norm of matrix ratio 3.73941e-007 difference 16.3073 Based on this it appears the Jacobian at the initial point is correct but 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 this indicates that the linear solver is not actually solving the linear system. One would expect a large drop in the "true resid norm" and one does not see that. This could be caused by 1) the computed Jacobian is wrong or 2) the Jacobian is singular and that is not being taken into account in the linear solver. Please rerun with the options -pc_type lu -ksp_monitor_true_residual -snes_monitor BUT NOT THE OPTION -snes_mf_operator and send the same output. Barry On May 11, 2011, at 1:46 AM, Tian(ICT) wrote: > Pls ignore my previous email. > > The followings are the test results: > the previous error in using -snes_mf_opertor was due to my own code. > After modification, i.e. using different A and B and assemble both > and in FormJacobian the native jacobian was assembled into B. > Then with -snes_mf_operator led to the results shown below. > The first three steps converged but with a diverged reason -6 at step 4. > > > With additional -snes_type test, the results were > >> "-pc_type >> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 9.05702e-008 difference 3.94971 >> Norm of matrix ratio 3.6015e-007 difference 15.7059 >> Norm of matrix ratio 3.73941e-007 difference 16.3073 >> [0]PETSC ERROR: SNESSolve() line 1871 in >> src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c >> [0]PETSC ERROR: User provided function() line 572 in >> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp >> [0]PETSC ERROR: User provided function() line 40 in >> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp >> >> >> "-pc_type >> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP Residual norm 9.965778978387e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm > 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.102368266281e-009 > 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm > 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 > 1 SNES Function norm 6.401926196865e-002 > 0 KSP Residual norm 8.969200166800e-002 > 0 KSP preconditioned resid norm 8.969200166800e-002 true resid norm > 6.401926196865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.492849351675e-009 > 1 KSP preconditioned resid norm 8.492849351675e-009 true resid norm > 1.400035213572e-002 ||Ae||/||Ax|| 2.186896834670e-001 > 2 SNES Function norm 5.849991037388e-002 > 0 KSP Residual norm 8.072279482667e-002 > 0 KSP preconditioned resid norm 8.072279482667e-002 true resid norm > 5.849991037388e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.610510067555e-009 > 1 KSP preconditioned resid norm 3.610510067555e-009 true resid norm > 4.757549946973e-003 ||Ae||/||Ax|| 8.132576471599e-002 > 3 SNES Function norm 5.376617142827e-002 > 0 KSP Residual norm 7.265050986128e-002 > 0 KSP preconditioned resid norm 7.265050986128e-002 true resid norm > 5.376617142827e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 6.941165647221e-009 > 1 KSP preconditioned resid norm 6.941165647221e-009 true resid norm > 3.935270526562e-002 ||Ae||/||Ax|| 7.319231446138e-001 > 4 SNES Function norm 4.956888929193e-002 > 0 KSP Residual norm 6.538545408372e-002 > 0 KSP preconditioned resid norm 6.538545408372e-002 true resid norm > 4.956888929193e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.457970649220e-009 > 1 KSP preconditioned resid norm 3.457970649220e-009 true resid norm > 2.037700110050e-002 ||Ae||/||Ax|| 4.110844804388e-001 > 5 SNES Function norm 4.575415001780e-002 > 0 KSP Residual norm 5.884690527647e-002 > 0 KSP preconditioned resid norm 5.884690527647e-002 true resid norm > 4.575415001780e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.832662187541e-009 > 1 KSP preconditioned resid norm 2.832662187541e-009 true resid norm > 1.662420124938e-002 ||Ae||/||Ax|| 3.633375604818e-001 > 6 SNES Function norm 4.223018337199e-002 > 0 KSP Residual norm 5.296221162333e-002 > 0 KSP preconditioned resid norm 5.296221162333e-002 true resid norm > 4.223018337199e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.091667005099e-009 > 1 KSP preconditioned resid norm 3.091667005099e-009 true resid norm > 2.235399010718e-002 ||Ae||/||Ax|| 5.293367994705e-001 > 7 SNES Function norm 3.894426990170e-002 > 0 KSP Residual norm 4.766598828506e-002 > 0 KSP preconditioned resid norm 4.766598828506e-002 true resid norm > 3.894426990170e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.223811830585e-009 > 1 KSP preconditioned resid norm 4.223811830585e-009 true resid norm > 2.831156780040e-002 ||Ae||/||Ax|| 7.269764684732e-001 > 8 SNES Function norm 3.586679674893e-002 > 0 KSP Residual norm 4.289938722959e-002 > 0 KSP preconditioned resid norm 4.289938722959e-002 true resid norm > 3.586679674893e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.504535460429e-009 > 1 KSP preconditioned resid norm 3.504535460429e-009 true resid norm > 2.254598873838e-002 ||Ae||/||Ax|| 6.286033541328e-001 > 9 SNES Function norm 3.298126871313e-002 > 0 KSP Residual norm 3.860944657267e-002 > 0 KSP preconditioned resid norm 3.860944657267e-002 true resid norm > 3.298126871313e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.439084585574e-009 > 1 KSP preconditioned resid norm 4.439084585574e-009 true resid norm > 6.936792391193e-003 ||Ae||/||Ax|| 2.103252137305e-001 > 10 SNES Function norm 3.027806037514e-002 > 0 KSP Residual norm 3.474850031738e-002 > 0 KSP preconditioned resid norm 3.474850031738e-002 true resid norm > 3.027806037514e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.476854671229e-009 > 1 KSP preconditioned resid norm 3.476854671229e-009 true resid norm > 1.935717696630e-002 ||Ae||/||Ax|| 6.393136391984e-001 > 11 SNES Function norm 2.775064646763e-002 > 0 KSP Residual norm 3.127364888698e-002 > 0 KSP preconditioned resid norm 3.127364888698e-002 true resid norm > 2.775064646763e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.380857715198e-009 > 1 KSP preconditioned resid norm 2.380857715198e-009 true resid norm > 1.646856223644e-002 ||Ae||/||Ax|| 5.934478771746e-001 > 12 SNES Function norm 2.430399371900e-002 > 0 KSP Residual norm 2.602654480862e-002 > 0 KSP preconditioned resid norm 2.602654480862e-002 true resid norm > 2.430399371900e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.556142726483e-009 > 1 KSP preconditioned resid norm 2.556142726483e-009 true resid norm > 1.794651743617e-002 ||Ae||/||Ax|| 7.384184526900e-001 > 13 SNES Function norm 2.029256664631e-002 > 0 KSP Residual norm 1.954645630691e-002 > 0 KSP preconditioned resid norm 1.954645630691e-002 true resid norm > 2.029256664631e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.913585412092e-010 > 1 KSP preconditioned resid norm 3.913585412092e-010 true resid norm > 1.903675728342e-003 ||Ae||/||Ax|| 9.381148089946e-002 > 14 SNES Function norm 1.566638821565e-002 > 0 KSP Residual norm 1.231404128086e-002 > 0 KSP preconditioned resid norm 1.231404128086e-002 true resid norm > 1.566638821565e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.258598971379e-010 > 1 KSP preconditioned resid norm 7.258598971379e-010 true resid norm > 1.221317129610e-003 ||Ae||/||Ax|| 7.795779810881e-002 > 15 SNES Function norm 1.064422587384e-002 > 0 KSP Residual norm 3.723145759360e-009 > 0 KSP preconditioned resid norm 3.723145759360e-009 true resid norm > 1.064422587384e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.546825086207e-016 > 1 KSP preconditioned resid norm 5.546825086207e-016 true resid norm > 2.984647735625e-009 ||Ae||/||Ax|| 2.804006389003e-007 > 16 SNES Function norm 9.689226075617e-008 > 0 KSP Residual norm 1.643080767133e-014 > 0 KSP preconditioned resid norm 1.643080767133e-014 true resid norm > 9.689226075617e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.291965935374e-022 > 1 KSP preconditioned resid norm 7.291965935374e-022 true resid norm > 5.001850432973e-015 ||Ae||/||Ax|| 5.162280654758e-008 > 17 SNES Function norm 3.747290584108e-008 > STEP 0 (Newton iterations: 17) > > 0 SNES Function norm 7.071067811866e-002 > 0 KSP Residual norm 9.965769844003e-002 > 0 KSP preconditioned resid norm 9.965769844003e-002 true resid norm > 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.919247724727e-009 > 1 KSP preconditioned resid norm 4.919247724727e-009 true resid norm > 2.073848008334e-002 ||Ae||/||Ax|| 2.932863979685e-001 > 1 SNES Function norm 6.402041310994e-002 > 0 KSP Residual norm 8.969192038029e-002 > 0 KSP preconditioned resid norm 8.969192038029e-002 true resid norm > 6.402041310994e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.072962844649e-009 > 1 KSP preconditioned resid norm 4.072962844649e-009 true resid norm > 2.116458841832e-002 ||Ae||/||Ax|| 3.305912503559e-001 > 2 SNES Function norm 5.850358277995e-002 > 0 KSP Residual norm 8.072272129834e-002 > 0 KSP preconditioned resid norm 8.072272129834e-002 true resid norm > 5.850358277995e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.535693836801e-009 > 1 KSP preconditioned resid norm 1.535693836801e-009 true resid norm > 1.378413836547e-002 ||Ae||/||Ax|| 2.356118670085e-001 > 3 SNES Function norm 5.377276379928e-002 > 0 KSP Residual norm 7.265044334812e-002 > 0 KSP preconditioned resid norm 7.265044334812e-002 true resid norm > 5.377276379928e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.196914267894e-009 > 1 KSP preconditioned resid norm 8.196914267894e-009 true resid norm > 1.643681592299e-002 ||Ae||/||Ax|| 3.056717706448e-001 > 4 SNES Function norm 4.957825627064e-002 > 0 KSP Residual norm 6.538539498475e-002 > 0 KSP preconditioned resid norm 6.538539498475e-002 true resid norm > 4.957825627064e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.605278143372e-009 > 1 KSP preconditioned resid norm 1.605278143372e-009 true resid norm > 1.365749144340e-002 ||Ae||/||Ax|| 2.754734125550e-001 > 5 SNES Function norm 4.576573391352e-002 > 0 KSP Residual norm 5.884685187863e-002 > 0 KSP preconditioned resid norm 5.884685187863e-002 true resid norm > 4.576573391352e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.917790130377e-009 > 1 KSP preconditioned resid norm 3.917790130377e-009 true resid norm > 2.918137698804e-002 ||Ae||/||Ax|| 6.376250196967e-001 > 6 SNES Function norm 4.224348383506e-002 > 0 KSP Residual norm 5.296216341013e-002 > 0 KSP preconditioned resid norm 5.296216341013e-002 true resid norm > 4.224348383506e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.384279752582e-009 > 1 KSP preconditioned resid norm 1.384279752582e-009 true resid norm > 8.645143617596e-003 ||Ae||/||Ax|| 2.046503468168e-001 > 7 SNES Function norm 3.895875122953e-002 > 0 KSP Residual norm 4.766594475257e-002 > 0 KSP preconditioned resid norm 4.766594475257e-002 true resid norm > 3.895875122953e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.442999105246e-009 > 1 KSP preconditioned resid norm 5.442999105246e-009 true resid norm > 3.199366262059e-002 ||Ae||/||Ax|| 8.212188946227e-001 > 8 SNES Function norm 3.588203219083e-002 > 0 KSP Residual norm 4.289934859308e-002 > 0 KSP preconditioned resid norm 4.289934859308e-002 true resid norm > 3.588203219083e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.069272581944e-009 > 1 KSP preconditioned resid norm 5.069272581944e-009 true resid norm > 3.315026802067e-002 ||Ae||/||Ax|| 9.238681868510e-001 > 9 SNES Function norm 3.299674865285e-002 > 0 KSP Residual norm 3.860941169595e-002 > 0 KSP preconditioned resid norm 3.860941169595e-002 true resid norm > 3.299674865285e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.314510318621e-009 > 1 KSP preconditioned resid norm 2.314510318621e-009 true resid norm > 1.422355468145e-002 ||Ae||/||Ax|| 4.310592789335e-001 > 10 SNES Function norm 3.020092421335e-002 > 0 KSP Residual norm 3.457428537321e-002 > 0 KSP preconditioned resid norm 3.457428537321e-002 true resid norm > 3.020092421335e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.171225245965e-009 > 1 KSP preconditioned resid norm 3.171225245965e-009 true resid norm > 6.700877909403e-003 ||Ae||/||Ax|| 2.218765843742e-001 > 11 SNES Function norm 2.761432326644e-002 > 0 KSP Residual norm 3.100402216444e-002 > 0 KSP preconditioned resid norm 3.100402216444e-002 true resid norm > 2.761432326644e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.317753799744e-009 > 1 KSP preconditioned resid norm 2.317753799744e-009 true resid norm > 1.143024510909e-002 ||Ae||/||Ax|| 4.139245057287e-001 > 12 SNES Function norm 2.483228477543e-002 > 0 KSP Residual norm 2.716039584199e-002 > 0 KSP preconditioned resid norm 2.716039584199e-002 true resid norm > 2.483228477543e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.024860175617e-009 > 1 KSP preconditioned resid norm 1.024860175617e-009 true resid norm > 5.071961436716e-003 ||Ae||/||Ax|| 2.042486820115e-001 > 13 SNES Function norm 2.155599009420e-002 > 0 KSP Residual norm 2.247739022940e-002 > 0 KSP preconditioned resid norm 2.247739022940e-002 true resid norm > 2.155599009420e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.231896568498e-009 > 1 KSP preconditioned resid norm 1.231896568498e-009 true resid norm > 2.253717888475e-003 ||Ae||/||Ax|| 1.045518150002e-001 > 14 SNES Function norm 1.758159262430e-002 > 0 KSP Residual norm 1.640654408633e-002 > 0 KSP preconditioned resid norm 1.640654408633e-002 true resid norm > 1.758159262430e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.203409491551e-009 > 1 KSP preconditioned resid norm 1.203409491551e-009 true resid norm > 3.340286893989e-003 ||Ae||/||Ax|| 1.899877312236e-001 > 15 SNES Function norm 1.275874713861e-002 > 0 KSP Residual norm 8.737890404296e-003 > 0 KSP preconditioned resid norm 8.737890404296e-003 true resid norm > 1.275874713861e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.862616851249e-010 > 1 KSP preconditioned resid norm 3.862616851249e-010 true resid norm > 9.158071400861e-004 ||Ae||/||Ax|| 7.177876715771e-002 > 16 SNES Function norm 5.367994969686e-003 > 0 KSP Residual norm 1.873266786098e-009 > 0 KSP preconditioned resid norm 1.873266786098e-009 true resid norm > 5.367994969686e-003 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 9.236612780542e-017 > 1 KSP preconditioned resid norm 9.236612780542e-017 true resid norm > 4.487656305545e-010 ||Ae||/||Ax|| 8.360023306445e-008 > 17 SNES Function norm 1.843858578272e-008 > 0 KSP Residual norm 3.418191528615e-015 > 0 KSP preconditioned resid norm 3.418191528615e-015 true resid norm > 1.843858578272e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.091028447697e-022 > 1 KSP preconditioned resid norm 2.091028447697e-022 true resid norm > 9.356082852780e-016 ||Ae||/||Ax|| 5.074186796662e-008 > 18 SNES Function norm 1.582175998776e-008 > STEP 1 (Newton iterations: 18) > > 0 SNES Function norm 7.071067811866e-002 > 0 KSP Residual norm 9.965760716669e-002 > 0 KSP preconditioned resid norm 9.965760716669e-002 true resid norm > 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.158690871520e-009 > 1 KSP preconditioned resid norm 4.158690871520e-009 true resid norm > 1.356168089543e-002 ||Ae||/||Ax|| 1.917911305090e-001 > 1 SNES Function norm 6.402155946870e-002 > 0 KSP Residual norm 8.969183775926e-002 > 0 KSP preconditioned resid norm 8.969183775926e-002 true resid norm > 6.402155946870e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.922035131300e-009 > 1 KSP preconditioned resid norm 7.922035131300e-009 true resid norm > 4.173737132742e-002 ||Ae||/||Ax|| 6.519268145573e-001 > 2 SNES Function norm 5.850726192206e-002 > 0 KSP Residual norm 8.072264774465e-002 > 0 KSP preconditioned resid norm 8.072264774465e-002 true resid norm > 5.850726192206e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.356474211542e-009 > 1 KSP preconditioned resid norm 4.356474211542e-009 true resid norm > 2.083500604436e-002 ||Ae||/||Ax|| 3.561097436437e-001 > 3 SNES Function norm 5.377933295280e-002 > 0 KSP Residual norm 7.265037686612e-002 > 0 KSP preconditioned resid norm 7.265037686612e-002 true resid norm > 5.377933295280e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.772318530630e-009 > 1 KSP preconditioned resid norm 8.772318530630e-009 true resid norm > 5.284958913103e-002 ||Ae||/||Ax|| 9.827118751623e-001 > 4 SNES Function norm 4.958751191600e-002 > 0 KSP Residual norm 6.538533386375e-002 > 0 KSP preconditioned resid norm 6.538533386375e-002 true resid norm > 4.958751191600e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.128741903850e-009 > 1 KSP preconditioned resid norm 1.128741903850e-009 true resid norm > 2.526410501148e-003 ||Ae||/||Ax|| 5.094852319729e-002 > 5 SNES Function norm 4.577729047509e-002 > 0 KSP Residual norm 5.884679667348e-002 > 0 KSP preconditioned resid norm 5.884679667348e-002 true resid norm > 4.577729047509e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.763627011382e-009 > 1 KSP preconditioned resid norm 5.763627011382e-009 true resid norm > 1.051243252216e-002 ||Ae||/||Ax|| 2.296429608013e-001 > 6 SNES Function norm 4.225677503639e-002 > 0 KSP Residual norm 5.296211439112e-002 > 0 KSP preconditioned resid norm 5.296211439112e-002 true resid norm > 4.225677503639e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.022912340341e-009 > 1 KSP preconditioned resid norm 1.022912340341e-009 true resid norm > 3.573383237647e-003 ||Ae||/||Ax|| 8.456355778617e-002 > 7 SNES Function norm 3.897321226706e-002 > 0 KSP Residual norm 4.766590050155e-002 > 0 KSP preconditioned resid norm 4.766590050155e-002 true resid norm > 3.897321226706e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.528025220977e-009 > 1 KSP preconditioned resid norm 3.528025220977e-009 true resid norm > 1.960313404416e-002 ||Ae||/||Ax|| 5.029899488355e-001 > 8 SNES Function norm 3.589720610234e-002 > 0 KSP Residual norm 4.289930864255e-002 > 0 KSP preconditioned resid norm 4.289930864255e-002 true resid norm > 3.589720610234e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 4.133432759880e-009 > 1 KSP preconditioned resid norm 4.133432759880e-009 true resid norm > 1.940005023620e-002 ||Ae||/||Ax|| 5.404334304149e-001 > 9 SNES Function norm 3.301230929196e-002 > 0 KSP Residual norm 3.860937624684e-002 > 0 KSP preconditioned resid norm 3.860937624684e-002 true resid norm > 3.301230929196e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.211304264406e-010 > 1 KSP preconditioned resid norm 8.211304264406e-010 true resid norm > 7.328958637952e-003 ||Ae||/||Ax|| 2.220068451781e-001 > 10 SNES Function norm 3.030900587812e-002 > 0 KSP Residual norm 3.474843739750e-002 > 0 KSP preconditioned resid norm 3.474843739750e-002 true resid norm > 3.030900587812e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.265840938498e-009 > 1 KSP preconditioned resid norm 3.265840938498e-009 true resid norm > 1.358870142093e-002 ||Ae||/||Ax|| 4.483387371916e-001 > 11 SNES Function norm 2.718409957830e-002 > 0 KSP Residual norm 3.013614572851e-002 > 0 KSP preconditioned resid norm 3.013614572851e-002 true resid norm > 2.718409957830e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.652744245097e-009 > 1 KSP preconditioned resid norm 1.652744245097e-009 true resid norm > 1.202105117910e-002 ||Ae||/||Ax|| 4.422089149754e-001 > 12 SNES Function norm 2.437348813670e-002 > 0 KSP Residual norm 2.629377332723e-002 > 0 KSP preconditioned resid norm 2.629377332723e-002 true resid norm > 2.437348813670e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 3.959255153953e-009 > 1 KSP preconditioned resid norm 3.959255153953e-009 true resid norm > 2.578989386428e-002 ||Ae||/||Ax|| 1.058112557367e+000 > 13 SNES Function norm 2.199784341474e-002 > 0 KSP Residual norm 2.328182605620e-002 > 0 KSP preconditioned resid norm 2.328182605620e-002 true resid norm > 2.199784341474e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 8.915626892590e-010 > 1 KSP preconditioned resid norm 8.915626892590e-010 true resid norm > 6.263373898192e-003 ||Ae||/||Ax|| 2.847267243477e-001 > 14 SNES Function norm 1.833996742657e-002 > 0 KSP Residual norm 1.794186796967e-002 > 0 KSP preconditioned resid norm 1.794186796967e-002 true resid norm > 1.833996742657e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 1.694980794482e-009 > 1 KSP preconditioned resid norm 1.694980794482e-009 true resid norm > 2.372076451714e-003 ||Ae||/||Ax|| 1.293391856453e-001 > 15 SNES Function norm 1.384131369872e-002 > 0 KSP Residual norm 1.074320273869e-002 > 0 KSP preconditioned resid norm 1.074320273869e-002 true resid norm > 1.384131369872e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 7.118408455169e-010 > 1 KSP preconditioned resid norm 7.118408455169e-010 true resid norm > 4.592913592019e-003 ||Ae||/||Ax|| 3.318264213927e-001 > 16 SNES Function norm 8.126551489387e-003 > 0 KSP Residual norm 2.817712852237e-009 > 0 KSP preconditioned resid norm 2.817712852237e-009 true resid norm > 8.126551489387e-003 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 2.239560797627e-016 > 1 KSP preconditioned resid norm 2.239560797627e-016 true resid norm > 1.053395037920e-009 ||Ae||/||Ax|| 1.296238680448e-007 > 17 SNES Function norm 4.871218193429e-008 > 0 KSP Residual norm 1.123675925695e-014 > 0 KSP preconditioned resid norm 1.123675925695e-014 true resid norm > 4.871218193429e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP Residual norm 5.307928330547e-022 > 1 KSP preconditioned resid norm 5.307928330547e-022 true resid norm > 2.893354456541e-015 ||Ae||/||Ax|| 5.939693813026e-008 > 18 SNES Function norm 5.035279029783e-008 > STEP 2 (Newton iterations: 18) > > diverged reason: -6 > > > Rong > >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Wednesday, May 11, 2011 11:05 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> Ok, the linear solver looks like it is working ok. The likely problem is >> that Jacobian does not match the function evaluation. >> >> Run the same thing but with the additional option -snes_mf_operator >> >> Then run with -snes_type test (instead of -snes_mf_operator). >> >> Barry >> >> On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: >> >>> Dear Barry, here is the output using -pc_type >>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>> the attached is the same and for clear reference. Thanks again for >>> helps. >>> >>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>> 0 SNES Function norm 7.071067811865e-002 >>> 0 KSP Residual norm 9.965778978387e-002 >>> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >>> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.823187455811e-017 >>> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >>> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >>> 1 SNES Function norm 6.401926523423e-002 >>> 0 KSP Residual norm 8.969200212486e-002 >>> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >>> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.106757475780e-016 >>> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >>> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >>> 2 SNES Function norm 5.849992149767e-002 >>> 0 KSP Residual norm 8.072279488157e-002 >>> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >>> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.268750073799e-017 >>> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >>> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >>> 3 SNES Function norm 5.376618503592e-002 >>> 0 KSP Residual norm 7.265050969883e-002 >>> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >>> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.677655733356e-017 >>> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >>> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >>> 4 SNES Function norm 4.956894354459e-002 >>> 0 KSP Residual norm 6.538545411661e-002 >>> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >>> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 9.557004153175e-017 >>> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >>> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >>> 5 SNES Function norm 4.575418613137e-002 >>> 0 KSP Residual norm 5.884690496914e-002 >>> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >>> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 5.470969262115e-017 >>> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >>> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >>> 6 SNES Function norm 4.223022245585e-002 >>> 0 KSP Residual norm 5.296221144636e-002 >>> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >>> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 8.255198782390e-017 >>> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >>> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >>> 7 SNES Function norm 3.894430065910e-002 >>> 0 KSP Residual norm 4.766598785088e-002 >>> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >>> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 3.322615478395e-017 >>> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >>> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >>> 8 SNES Function norm 3.586683371135e-002 >>> 0 KSP Residual norm 4.289938708067e-002 >>> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >>> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.181358328498e-017 >>> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >>> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >>> 9 SNES Function norm 3.298130202025e-002 >>> 0 KSP Residual norm 3.860944676473e-002 >>> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >>> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 4.635174776374e-017 >>> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >>> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >>> 10 SNES Function norm 3.027806208930e-002 >>> 0 KSP Residual norm 3.474850078591e-002 >>> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >>> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.389914053685e-017 >>> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >>> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >>> 11 SNES Function norm 2.749422924729e-002 >>> 0 KSP Residual norm 3.081350823297e-002 >>> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >>> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.640567497647e-017 >>> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >>> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >>> 12 SNES Function norm 2.437488247885e-002 >>> 0 KSP Residual norm 2.633007441879e-002 >>> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >>> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.772331460094e-017 >>> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >>> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >>> 13 SNES Function norm 2.079664278637e-002 >>> 0 KSP Residual norm 2.104738289397e-002 >>> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >>> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.650632708670e-017 >>> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >>> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >>> 14 SNES Function norm 1.657344626858e-002 >>> 0 KSP Residual norm 1.454141853505e-002 >>> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >>> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 1.129401160070e-017 >>> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >>> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >>> 15 SNES Function norm 1.484243752612e-002 >>> 0 KSP Residual norm 5.241948491751e-009 >>> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >>> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 2.729506849025e-024 >>> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >>> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >>> 16 SNES Function norm 2.828002157497e-008 >>> 0 KSP Residual norm 6.042518362322e-015 >>> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >>> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >>> 1 KSP Residual norm 6.272441346127e-030 >>> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >>> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >>> 17 SNES Function norm 2.960967020289e-008 >>> STEP 0 (Newton iterations: 17) >>> >>> diverged reason: -6 >>> >>> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Wednesday, May 11, 2011 2:54 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>> and >>> send the outputs >>> >>> >>> Barry >>> >>> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >>> >>>> by the way, the increment size is like that >>>> for a 100 lengh model, the increment is set to 0.05, >>>> the engineering strain is around 5%% per load step. >>>> This is already too small increment size for a large deformation >>>> analysis. >>>> a 0.5 increment size leads to both linear search and trust region >>>> failed. >>>> linear search failed for 0.05 while trust region converges with 17 >>>> Newton >>>> iterations each load step. >>>> Rong >>>> >>>> ----- Original Message ----- From: "Tian(ICT)" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 11:37 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>>> First, thanks again, the issue was gone. >>>>> >>>>> I just followed up with some test results. >>>>> I have tested SNES using one finite element for a geometric large >>>>> deformation problem. >>>>> Those are just the very early test results so they may be not telling >>>>> what happened exactly. >>>>> For the displacement controlled load, I found that convergence is much >>>>> slower than that of force loading. >>>>> Even worse, linear search is so sensitive to the displacement >>>>> increment >>>>> and diverged no matter what the increment size was used (too small >>>>> incremnt also led to diverged soloution (-6 reason), trust region >>>>> works >>>>> well in the sense of not sensitive to the displacement increment, but >>>>> during each load step, it requires around ten to several tens of >>>>> Newton >>>>> interations whereas for the force loading case and the almost same >>>>> amount of deformation, this is normally 3. This is against my >>>>> expectation. Any hint? >>>>> >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Barry Smith" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>> >>>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>>> >>>>>> Dear Barry, Thanks a lot for quick answering. >>>>>> I checked the development documents and found the new version of >>>>>> MatZeroRows() does support the nonzero prescribed boundary >>>>>> conditions. >>>>>> >>>>>> I followed up with more details. >>>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using >>>>>> SNES >>>>>> solvers. >>>>>> I used a displacement-controlled load (as this type of loading works >>>>>> well for all cases). >>>>>> This is the reason the nonzero prescribed boundary came up. >>>>>> >>>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>>> nonzero prescribed boundary. >>>>>> In FormFunction, I modified the solution to the known solution(this >>>>>> should not be necessary as the modified Jacobian and rhs should give >>>>>> the prescribed solution also) >>>>> >>>>> You should not do it this way. See below. >>>>>> >>>>>> Now I found another issue, no matter if I prescried the solution or >>>>>> not >>>>>> in FormFunction, >>>>>> SNES solver always call FormFunction and never call FormJacobian. >>>>> >>>>> The only reason it would not call FormJacobian is if decided that the >>>>> residual norm was small enough before any Newton steps; for example if >>>>> the FormFunction() computed exactly the zero function initially. When >>>>> you run with -snes_monitor -ksp_monitor what does it print for >>>>> residual >>>>> norms. >>>>> >>>>>> Of course the solver finally diverged or converged to a zero >>>>>> solution. >>>>>> >>>>>> So my quick follow up question is How a displacement-controled load >>>>>> is >>>>>> done corrently in Petsc 2.3.3? >>>>> >>>>> To do it in 2.3.3 simply have for those components of F() the formula >>>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() >>>>> for >>>>> those rows >>>>> >>>>> We strongly urge you to upgrade to the latest PETSc before doing >>>>> anything further. >>>>> >>>>> >>>>> Barry >>>>> >>>>>> >>>>>> Rong >>>>>> >>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>> To: "PETSc users list" >>>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>> >>>>>> >>>>>> >>>>>> In petsc-dev >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>>> modified the calling sequence for MatZeroRows() so that it can >>>>>> automatically adjust the appropriate right hand side values for the >>>>>> zeroed rows to support zero or non-zero prescribed boundary >>>>>> conditions >>>>>> easily. >>>>>> >>>>>> Barry >>>>>> >>>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>>> >>>>>>> Dear all, >>>>>>> >>>>>>> I got this question long ago and searched the prior posting but did >>>>>>> not find the solution. >>>>>>> The question is about nonzero prescribed boundary condition. >>>>>>> My understanding is that MatZeroRows() works only for zero >>>>>>> prescribed >>>>>>> value, not non-zero value. >>>>>>> For the non-zero values, we have to remove the rows associated with >>>>>>> the boundary, but this >>>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>>> also be removed. >>>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>>> prescribed boundary and if so how to do it simply? >>>>>>> >>>>>>> Rong >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> > From jed at 59A2.org Wed May 11 11:02:25 2011 From: jed at 59A2.org (Jed Brown) Date: Wed, 11 May 2011 18:02:25 +0200 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: On Wed, May 11, 2011 at 04:20, Sylvain Barbot wrote: > I am still trying to design a > multigrid preconditionner for the Navier's equation of elasticity. > I have heard, through an external source, that you have large jumps in both Young's modulus and Poisson ratio that are not grid aligned, including perhaps thin structures that span a large part of the domain. Such problems are pretty hard, so I suggest you focus on robustness and do not worry about low-memory implementation at this point. That is, you should assemble the matrices in a usual PETSc format instead of using MatShell to do everything matrix-free. This gives you access to much stronger smoothers. After you find a scheme that is robust enough for your purposes, _then_ you can make it low-memory by replacing some assembled matrices by MatShell. To realize most of the possible memory savings, it should be sufficient to do this on the finest level only. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed May 11 11:15:57 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 11 May 2011 11:15:57 -0500 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <32DB8456522B4FABB725933C113D66D1@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop><1302D684D3644298A3F2528BDEE08857@rti458laptop> <32DB8456522B4FABB725933C113D66D1@rti458laptop> Message-ID: <03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov> Ok, Run with -pc_type none -ksp_monitor_true_residual -snes_monitor -snes_mf Send the output Barry On May 11, 2011, at 8:44 AM, Tian(ICT) wrote: > C:\cygwin\home\rabbit\pGFEM-test>pgfem -pc_type lu -ksp_monitor_true_residual -s > nes_monitor > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.07106781 > 1865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm 8.84729888 > 5656e-011 ||Ae||/||Ax|| 1.251197007446e-009 > 1 SNES Function norm 6.401926523423e-002 > 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm 6.40192652 > 3423e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm 6.21183006 > 7439e-011 ||Ae||/||Ax|| 9.703063671087e-010 > 2 SNES Function norm 5.849992149767e-002 > 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm 5.84999214 > 9767e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm 3.80243103 > 6387e-011 ||Ae||/||Ax|| 6.499890835816e-010 > 3 SNES Function norm 5.376618503592e-002 > 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm 5.37661850 > 3592e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm 8.12039778 > 8686e-011 ||Ae||/||Ax|| 1.510316899602e-009 > 4 SNES Function norm 4.956894354459e-002 > 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm 4.95689435 > 4459e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm 2.94425080 > 2029e-011 ||Ae||/||Ax|| 5.939708598754e-010 > 5 SNES Function norm 4.575418613137e-002 > 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm 4.57541861 > 3137e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm 3.65900316 > 6095e-011 ||Ae||/||Ax|| 7.997089393284e-010 > 6 SNES Function norm 4.223022245585e-002 > 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm 4.22302224 > 5585e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm 1.95554565 > 8933e-011 ||Ae||/||Ax|| 4.630678090739e-010 > 7 SNES Function norm 3.894430065910e-002 > 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm 3.89443006 > 5910e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm 3.48532814 > 8673e-011 ||Ae||/||Ax|| 8.949520442496e-010 > 8 SNES Function norm 3.586683371135e-002 > 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm 3.58668337 > 1135e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm 3.24690281 > 8086e-011 ||Ae||/||Ax|| 9.052660862724e-010 > 9 SNES Function norm 3.298130202025e-002 > 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm 3.29813020 > 2025e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm 1.49751684 > 2272e-011 ||Ae||/||Ax|| 4.540502498513e-010 > 10 SNES Function norm 3.027806208930e-002 > 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm 3.02780620 > 8930e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm 6.00744088 > 8596e-011 ||Ae||/||Ax|| 1.984090286517e-009 > 11 SNES Function norm 2.749422924729e-002 > 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm 2.74942292 > 4729e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm 1.28163829 > 5853e-011 ||Ae||/||Ax|| 4.661481085089e-010 > 12 SNES Function norm 2.437488247885e-002 > 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm 2.43748824 > 7885e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm 1.91821249 > 6143e-011 ||Ae||/||Ax|| 7.869627670236e-010 > 13 SNES Function norm 2.079664278637e-002 > 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm 2.07966427 > 8637e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm 2.31637196 > 7362e-011 ||Ae||/||Ax|| 1.113820144509e-009 > 14 SNES Function norm 1.657344626858e-002 > 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm 1.65734462 > 6858e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm 7.88549932 > 7559e-012 ||Ae||/||Ax|| 4.757911661686e-010 > 15 SNES Function norm 1.484243752612e-002 > 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm 1.48424375 > 2612e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm 6.38667785 > 1085e-018 ||Ae||/||Ax|| 4.302984492839e-016 > 16 SNES Function norm 2.828002157497e-008 > 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm 2.82800215 > 7497e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm 1.11285769 > 8032e-023 ||Ae||/||Ax|| 3.935137372797e-016 > 17 SNES Function norm 2.960967020289e-008 > STEP 0 (Newton iterations: 17) > > diverged reason: -6 > > C:\cygwin\home\rabbit\pGFEM-test> > > > ==================HOWEVER================= > -snes_type tr seems work > > C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_type tr > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > STEP 0 (Newton iterations: 12) > > STEP 1 (Newton iterations: 11) > > STEP 2 (Newton iterations: 11) > ... > > STEP 70 (Newton iterations: 14) > > ^C > C:\cygwin\home\rabbit\pGFEM-test> > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 9:02 PM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > >>> Testing hand-coded Jacobian, if the ratio is >>> O(1.e-8), the hand-coded Jacobian is probably correct. >>> Run with -snes_test_display to show difference >>> of hand-coded and finite difference Jacobian. >>> Norm of matrix ratio 9.05702e-008 difference 3.94971 >>> Norm of matrix ratio 3.6015e-007 difference 15.7059 >>> Norm of matrix ratio 3.73941e-007 difference 16.3073 > > Based on this it appears the Jacobian at the initial point is correct but > > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 > > this indicates that the linear solver is not actually solving the linear system. One would expect a large drop in the "true resid norm" and one does not see that. This could be caused by > > 1) the computed Jacobian is wrong or > > 2) the Jacobian is singular and that is not being taken into account in the linear solver. > > Please rerun with the options -pc_type lu -ksp_monitor_true_residual -snes_monitor BUT NOT THE OPTION -snes_mf_operator and send the same output. > > Barry > > > On May 11, 2011, at 1:46 AM, Tian(ICT) wrote: > >> Pls ignore my previous email. >> >> The followings are the test results: >> the previous error in using -snes_mf_opertor was due to my own code. >> After modification, i.e. using different A and B and assemble both >> and in FormJacobian the native jacobian was assembled into B. >> Then with -snes_mf_operator led to the results shown below. >> The first three steps converged but with a diverged reason -6 at step 4. >> >> >> With additional -snes_type test, the results were >> >>> "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" >>> >>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>> Testing hand-coded Jacobian, if the ratio is >>> O(1.e-8), the hand-coded Jacobian is probably correct. >>> Run with -snes_test_display to show difference >>> of hand-coded and finite difference Jacobian. >>> Norm of matrix ratio 9.05702e-008 difference 3.94971 >>> Norm of matrix ratio 3.6015e-007 difference 15.7059 >>> Norm of matrix ratio 3.73941e-007 difference 16.3073 >>> [0]PETSC ERROR: SNESSolve() line 1871 in >>> src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c >>> [0]PETSC ERROR: User provided function() line 572 in >>> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp >>> [0]PETSC ERROR: User provided function() line 40 in >>> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp >>> >>> >>> "-pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> 0 SNES Function norm 7.071067811865e-002 >> 0 KSP Residual norm 9.965778978387e-002 >> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.102368266281e-009 >> 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 >> 1 SNES Function norm 6.401926196865e-002 >> 0 KSP Residual norm 8.969200166800e-002 >> 0 KSP preconditioned resid norm 8.969200166800e-002 true resid norm 6.401926196865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.492849351675e-009 >> 1 KSP preconditioned resid norm 8.492849351675e-009 true resid norm 1.400035213572e-002 ||Ae||/||Ax|| 2.186896834670e-001 >> 2 SNES Function norm 5.849991037388e-002 >> 0 KSP Residual norm 8.072279482667e-002 >> 0 KSP preconditioned resid norm 8.072279482667e-002 true resid norm 5.849991037388e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.610510067555e-009 >> 1 KSP preconditioned resid norm 3.610510067555e-009 true resid norm 4.757549946973e-003 ||Ae||/||Ax|| 8.132576471599e-002 >> 3 SNES Function norm 5.376617142827e-002 >> 0 KSP Residual norm 7.265050986128e-002 >> 0 KSP preconditioned resid norm 7.265050986128e-002 true resid norm 5.376617142827e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.941165647221e-009 >> 1 KSP preconditioned resid norm 6.941165647221e-009 true resid norm 3.935270526562e-002 ||Ae||/||Ax|| 7.319231446138e-001 >> 4 SNES Function norm 4.956888929193e-002 >> 0 KSP Residual norm 6.538545408372e-002 >> 0 KSP preconditioned resid norm 6.538545408372e-002 true resid norm 4.956888929193e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.457970649220e-009 >> 1 KSP preconditioned resid norm 3.457970649220e-009 true resid norm 2.037700110050e-002 ||Ae||/||Ax|| 4.110844804388e-001 >> 5 SNES Function norm 4.575415001780e-002 >> 0 KSP Residual norm 5.884690527647e-002 >> 0 KSP preconditioned resid norm 5.884690527647e-002 true resid norm 4.575415001780e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.832662187541e-009 >> 1 KSP preconditioned resid norm 2.832662187541e-009 true resid norm 1.662420124938e-002 ||Ae||/||Ax|| 3.633375604818e-001 >> 6 SNES Function norm 4.223018337199e-002 >> 0 KSP Residual norm 5.296221162333e-002 >> 0 KSP preconditioned resid norm 5.296221162333e-002 true resid norm 4.223018337199e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.091667005099e-009 >> 1 KSP preconditioned resid norm 3.091667005099e-009 true resid norm 2.235399010718e-002 ||Ae||/||Ax|| 5.293367994705e-001 >> 7 SNES Function norm 3.894426990170e-002 >> 0 KSP Residual norm 4.766598828506e-002 >> 0 KSP preconditioned resid norm 4.766598828506e-002 true resid norm 3.894426990170e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.223811830585e-009 >> 1 KSP preconditioned resid norm 4.223811830585e-009 true resid norm 2.831156780040e-002 ||Ae||/||Ax|| 7.269764684732e-001 >> 8 SNES Function norm 3.586679674893e-002 >> 0 KSP Residual norm 4.289938722959e-002 >> 0 KSP preconditioned resid norm 4.289938722959e-002 true resid norm 3.586679674893e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.504535460429e-009 >> 1 KSP preconditioned resid norm 3.504535460429e-009 true resid norm 2.254598873838e-002 ||Ae||/||Ax|| 6.286033541328e-001 >> 9 SNES Function norm 3.298126871313e-002 >> 0 KSP Residual norm 3.860944657267e-002 >> 0 KSP preconditioned resid norm 3.860944657267e-002 true resid norm 3.298126871313e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.439084585574e-009 >> 1 KSP preconditioned resid norm 4.439084585574e-009 true resid norm 6.936792391193e-003 ||Ae||/||Ax|| 2.103252137305e-001 >> 10 SNES Function norm 3.027806037514e-002 >> 0 KSP Residual norm 3.474850031738e-002 >> 0 KSP preconditioned resid norm 3.474850031738e-002 true resid norm 3.027806037514e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.476854671229e-009 >> 1 KSP preconditioned resid norm 3.476854671229e-009 true resid norm 1.935717696630e-002 ||Ae||/||Ax|| 6.393136391984e-001 >> 11 SNES Function norm 2.775064646763e-002 >> 0 KSP Residual norm 3.127364888698e-002 >> 0 KSP preconditioned resid norm 3.127364888698e-002 true resid norm 2.775064646763e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.380857715198e-009 >> 1 KSP preconditioned resid norm 2.380857715198e-009 true resid norm 1.646856223644e-002 ||Ae||/||Ax|| 5.934478771746e-001 >> 12 SNES Function norm 2.430399371900e-002 >> 0 KSP Residual norm 2.602654480862e-002 >> 0 KSP preconditioned resid norm 2.602654480862e-002 true resid norm 2.430399371900e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.556142726483e-009 >> 1 KSP preconditioned resid norm 2.556142726483e-009 true resid norm 1.794651743617e-002 ||Ae||/||Ax|| 7.384184526900e-001 >> 13 SNES Function norm 2.029256664631e-002 >> 0 KSP Residual norm 1.954645630691e-002 >> 0 KSP preconditioned resid norm 1.954645630691e-002 true resid norm 2.029256664631e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.913585412092e-010 >> 1 KSP preconditioned resid norm 3.913585412092e-010 true resid norm 1.903675728342e-003 ||Ae||/||Ax|| 9.381148089946e-002 >> 14 SNES Function norm 1.566638821565e-002 >> 0 KSP Residual norm 1.231404128086e-002 >> 0 KSP preconditioned resid norm 1.231404128086e-002 true resid norm 1.566638821565e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.258598971379e-010 >> 1 KSP preconditioned resid norm 7.258598971379e-010 true resid norm 1.221317129610e-003 ||Ae||/||Ax|| 7.795779810881e-002 >> 15 SNES Function norm 1.064422587384e-002 >> 0 KSP Residual norm 3.723145759360e-009 >> 0 KSP preconditioned resid norm 3.723145759360e-009 true resid norm 1.064422587384e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.546825086207e-016 >> 1 KSP preconditioned resid norm 5.546825086207e-016 true resid norm 2.984647735625e-009 ||Ae||/||Ax|| 2.804006389003e-007 >> 16 SNES Function norm 9.689226075617e-008 >> 0 KSP Residual norm 1.643080767133e-014 >> 0 KSP preconditioned resid norm 1.643080767133e-014 true resid norm 9.689226075617e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.291965935374e-022 >> 1 KSP preconditioned resid norm 7.291965935374e-022 true resid norm 5.001850432973e-015 ||Ae||/||Ax|| 5.162280654758e-008 >> 17 SNES Function norm 3.747290584108e-008 >> STEP 0 (Newton iterations: 17) >> >> 0 SNES Function norm 7.071067811866e-002 >> 0 KSP Residual norm 9.965769844003e-002 >> 0 KSP preconditioned resid norm 9.965769844003e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.919247724727e-009 >> 1 KSP preconditioned resid norm 4.919247724727e-009 true resid norm 2.073848008334e-002 ||Ae||/||Ax|| 2.932863979685e-001 >> 1 SNES Function norm 6.402041310994e-002 >> 0 KSP Residual norm 8.969192038029e-002 >> 0 KSP preconditioned resid norm 8.969192038029e-002 true resid norm 6.402041310994e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.072962844649e-009 >> 1 KSP preconditioned resid norm 4.072962844649e-009 true resid norm 2.116458841832e-002 ||Ae||/||Ax|| 3.305912503559e-001 >> 2 SNES Function norm 5.850358277995e-002 >> 0 KSP Residual norm 8.072272129834e-002 >> 0 KSP preconditioned resid norm 8.072272129834e-002 true resid norm 5.850358277995e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.535693836801e-009 >> 1 KSP preconditioned resid norm 1.535693836801e-009 true resid norm 1.378413836547e-002 ||Ae||/||Ax|| 2.356118670085e-001 >> 3 SNES Function norm 5.377276379928e-002 >> 0 KSP Residual norm 7.265044334812e-002 >> 0 KSP preconditioned resid norm 7.265044334812e-002 true resid norm 5.377276379928e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.196914267894e-009 >> 1 KSP preconditioned resid norm 8.196914267894e-009 true resid norm 1.643681592299e-002 ||Ae||/||Ax|| 3.056717706448e-001 >> 4 SNES Function norm 4.957825627064e-002 >> 0 KSP Residual norm 6.538539498475e-002 >> 0 KSP preconditioned resid norm 6.538539498475e-002 true resid norm 4.957825627064e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.605278143372e-009 >> 1 KSP preconditioned resid norm 1.605278143372e-009 true resid norm 1.365749144340e-002 ||Ae||/||Ax|| 2.754734125550e-001 >> 5 SNES Function norm 4.576573391352e-002 >> 0 KSP Residual norm 5.884685187863e-002 >> 0 KSP preconditioned resid norm 5.884685187863e-002 true resid norm 4.576573391352e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.917790130377e-009 >> 1 KSP preconditioned resid norm 3.917790130377e-009 true resid norm 2.918137698804e-002 ||Ae||/||Ax|| 6.376250196967e-001 >> 6 SNES Function norm 4.224348383506e-002 >> 0 KSP Residual norm 5.296216341013e-002 >> 0 KSP preconditioned resid norm 5.296216341013e-002 true resid norm 4.224348383506e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.384279752582e-009 >> 1 KSP preconditioned resid norm 1.384279752582e-009 true resid norm 8.645143617596e-003 ||Ae||/||Ax|| 2.046503468168e-001 >> 7 SNES Function norm 3.895875122953e-002 >> 0 KSP Residual norm 4.766594475257e-002 >> 0 KSP preconditioned resid norm 4.766594475257e-002 true resid norm 3.895875122953e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.442999105246e-009 >> 1 KSP preconditioned resid norm 5.442999105246e-009 true resid norm 3.199366262059e-002 ||Ae||/||Ax|| 8.212188946227e-001 >> 8 SNES Function norm 3.588203219083e-002 >> 0 KSP Residual norm 4.289934859308e-002 >> 0 KSP preconditioned resid norm 4.289934859308e-002 true resid norm 3.588203219083e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.069272581944e-009 >> 1 KSP preconditioned resid norm 5.069272581944e-009 true resid norm 3.315026802067e-002 ||Ae||/||Ax|| 9.238681868510e-001 >> 9 SNES Function norm 3.299674865285e-002 >> 0 KSP Residual norm 3.860941169595e-002 >> 0 KSP preconditioned resid norm 3.860941169595e-002 true resid norm 3.299674865285e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.314510318621e-009 >> 1 KSP preconditioned resid norm 2.314510318621e-009 true resid norm 1.422355468145e-002 ||Ae||/||Ax|| 4.310592789335e-001 >> 10 SNES Function norm 3.020092421335e-002 >> 0 KSP Residual norm 3.457428537321e-002 >> 0 KSP preconditioned resid norm 3.457428537321e-002 true resid norm 3.020092421335e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.171225245965e-009 >> 1 KSP preconditioned resid norm 3.171225245965e-009 true resid norm 6.700877909403e-003 ||Ae||/||Ax|| 2.218765843742e-001 >> 11 SNES Function norm 2.761432326644e-002 >> 0 KSP Residual norm 3.100402216444e-002 >> 0 KSP preconditioned resid norm 3.100402216444e-002 true resid norm 2.761432326644e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.317753799744e-009 >> 1 KSP preconditioned resid norm 2.317753799744e-009 true resid norm 1.143024510909e-002 ||Ae||/||Ax|| 4.139245057287e-001 >> 12 SNES Function norm 2.483228477543e-002 >> 0 KSP Residual norm 2.716039584199e-002 >> 0 KSP preconditioned resid norm 2.716039584199e-002 true resid norm 2.483228477543e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.024860175617e-009 >> 1 KSP preconditioned resid norm 1.024860175617e-009 true resid norm 5.071961436716e-003 ||Ae||/||Ax|| 2.042486820115e-001 >> 13 SNES Function norm 2.155599009420e-002 >> 0 KSP Residual norm 2.247739022940e-002 >> 0 KSP preconditioned resid norm 2.247739022940e-002 true resid norm 2.155599009420e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.231896568498e-009 >> 1 KSP preconditioned resid norm 1.231896568498e-009 true resid norm 2.253717888475e-003 ||Ae||/||Ax|| 1.045518150002e-001 >> 14 SNES Function norm 1.758159262430e-002 >> 0 KSP Residual norm 1.640654408633e-002 >> 0 KSP preconditioned resid norm 1.640654408633e-002 true resid norm 1.758159262430e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.203409491551e-009 >> 1 KSP preconditioned resid norm 1.203409491551e-009 true resid norm 3.340286893989e-003 ||Ae||/||Ax|| 1.899877312236e-001 >> 15 SNES Function norm 1.275874713861e-002 >> 0 KSP Residual norm 8.737890404296e-003 >> 0 KSP preconditioned resid norm 8.737890404296e-003 true resid norm 1.275874713861e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.862616851249e-010 >> 1 KSP preconditioned resid norm 3.862616851249e-010 true resid norm 9.158071400861e-004 ||Ae||/||Ax|| 7.177876715771e-002 >> 16 SNES Function norm 5.367994969686e-003 >> 0 KSP Residual norm 1.873266786098e-009 >> 0 KSP preconditioned resid norm 1.873266786098e-009 true resid norm 5.367994969686e-003 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.236612780542e-017 >> 1 KSP preconditioned resid norm 9.236612780542e-017 true resid norm 4.487656305545e-010 ||Ae||/||Ax|| 8.360023306445e-008 >> 17 SNES Function norm 1.843858578272e-008 >> 0 KSP Residual norm 3.418191528615e-015 >> 0 KSP preconditioned resid norm 3.418191528615e-015 true resid norm 1.843858578272e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.091028447697e-022 >> 1 KSP preconditioned resid norm 2.091028447697e-022 true resid norm 9.356082852780e-016 ||Ae||/||Ax|| 5.074186796662e-008 >> 18 SNES Function norm 1.582175998776e-008 >> STEP 1 (Newton iterations: 18) >> >> 0 SNES Function norm 7.071067811866e-002 >> 0 KSP Residual norm 9.965760716669e-002 >> 0 KSP preconditioned resid norm 9.965760716669e-002 true resid norm 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.158690871520e-009 >> 1 KSP preconditioned resid norm 4.158690871520e-009 true resid norm 1.356168089543e-002 ||Ae||/||Ax|| 1.917911305090e-001 >> 1 SNES Function norm 6.402155946870e-002 >> 0 KSP Residual norm 8.969183775926e-002 >> 0 KSP preconditioned resid norm 8.969183775926e-002 true resid norm 6.402155946870e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.922035131300e-009 >> 1 KSP preconditioned resid norm 7.922035131300e-009 true resid norm 4.173737132742e-002 ||Ae||/||Ax|| 6.519268145573e-001 >> 2 SNES Function norm 5.850726192206e-002 >> 0 KSP Residual norm 8.072264774465e-002 >> 0 KSP preconditioned resid norm 8.072264774465e-002 true resid norm 5.850726192206e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.356474211542e-009 >> 1 KSP preconditioned resid norm 4.356474211542e-009 true resid norm 2.083500604436e-002 ||Ae||/||Ax|| 3.561097436437e-001 >> 3 SNES Function norm 5.377933295280e-002 >> 0 KSP Residual norm 7.265037686612e-002 >> 0 KSP preconditioned resid norm 7.265037686612e-002 true resid norm 5.377933295280e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.772318530630e-009 >> 1 KSP preconditioned resid norm 8.772318530630e-009 true resid norm 5.284958913103e-002 ||Ae||/||Ax|| 9.827118751623e-001 >> 4 SNES Function norm 4.958751191600e-002 >> 0 KSP Residual norm 6.538533386375e-002 >> 0 KSP preconditioned resid norm 6.538533386375e-002 true resid norm 4.958751191600e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.128741903850e-009 >> 1 KSP preconditioned resid norm 1.128741903850e-009 true resid norm 2.526410501148e-003 ||Ae||/||Ax|| 5.094852319729e-002 >> 5 SNES Function norm 4.577729047509e-002 >> 0 KSP Residual norm 5.884679667348e-002 >> 0 KSP preconditioned resid norm 5.884679667348e-002 true resid norm 4.577729047509e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.763627011382e-009 >> 1 KSP preconditioned resid norm 5.763627011382e-009 true resid norm 1.051243252216e-002 ||Ae||/||Ax|| 2.296429608013e-001 >> 6 SNES Function norm 4.225677503639e-002 >> 0 KSP Residual norm 5.296211439112e-002 >> 0 KSP preconditioned resid norm 5.296211439112e-002 true resid norm 4.225677503639e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.022912340341e-009 >> 1 KSP preconditioned resid norm 1.022912340341e-009 true resid norm 3.573383237647e-003 ||Ae||/||Ax|| 8.456355778617e-002 >> 7 SNES Function norm 3.897321226706e-002 >> 0 KSP Residual norm 4.766590050155e-002 >> 0 KSP preconditioned resid norm 4.766590050155e-002 true resid norm 3.897321226706e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.528025220977e-009 >> 1 KSP preconditioned resid norm 3.528025220977e-009 true resid norm 1.960313404416e-002 ||Ae||/||Ax|| 5.029899488355e-001 >> 8 SNES Function norm 3.589720610234e-002 >> 0 KSP Residual norm 4.289930864255e-002 >> 0 KSP preconditioned resid norm 4.289930864255e-002 true resid norm 3.589720610234e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.133432759880e-009 >> 1 KSP preconditioned resid norm 4.133432759880e-009 true resid norm 1.940005023620e-002 ||Ae||/||Ax|| 5.404334304149e-001 >> 9 SNES Function norm 3.301230929196e-002 >> 0 KSP Residual norm 3.860937624684e-002 >> 0 KSP preconditioned resid norm 3.860937624684e-002 true resid norm 3.301230929196e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.211304264406e-010 >> 1 KSP preconditioned resid norm 8.211304264406e-010 true resid norm 7.328958637952e-003 ||Ae||/||Ax|| 2.220068451781e-001 >> 10 SNES Function norm 3.030900587812e-002 >> 0 KSP Residual norm 3.474843739750e-002 >> 0 KSP preconditioned resid norm 3.474843739750e-002 true resid norm 3.030900587812e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.265840938498e-009 >> 1 KSP preconditioned resid norm 3.265840938498e-009 true resid norm 1.358870142093e-002 ||Ae||/||Ax|| 4.483387371916e-001 >> 11 SNES Function norm 2.718409957830e-002 >> 0 KSP Residual norm 3.013614572851e-002 >> 0 KSP preconditioned resid norm 3.013614572851e-002 true resid norm 2.718409957830e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.652744245097e-009 >> 1 KSP preconditioned resid norm 1.652744245097e-009 true resid norm 1.202105117910e-002 ||Ae||/||Ax|| 4.422089149754e-001 >> 12 SNES Function norm 2.437348813670e-002 >> 0 KSP Residual norm 2.629377332723e-002 >> 0 KSP preconditioned resid norm 2.629377332723e-002 true resid norm 2.437348813670e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.959255153953e-009 >> 1 KSP preconditioned resid norm 3.959255153953e-009 true resid norm 2.578989386428e-002 ||Ae||/||Ax|| 1.058112557367e+000 >> 13 SNES Function norm 2.199784341474e-002 >> 0 KSP Residual norm 2.328182605620e-002 >> 0 KSP preconditioned resid norm 2.328182605620e-002 true resid norm 2.199784341474e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.915626892590e-010 >> 1 KSP preconditioned resid norm 8.915626892590e-010 true resid norm 6.263373898192e-003 ||Ae||/||Ax|| 2.847267243477e-001 >> 14 SNES Function norm 1.833996742657e-002 >> 0 KSP Residual norm 1.794186796967e-002 >> 0 KSP preconditioned resid norm 1.794186796967e-002 true resid norm 1.833996742657e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.694980794482e-009 >> 1 KSP preconditioned resid norm 1.694980794482e-009 true resid norm 2.372076451714e-003 ||Ae||/||Ax|| 1.293391856453e-001 >> 15 SNES Function norm 1.384131369872e-002 >> 0 KSP Residual norm 1.074320273869e-002 >> 0 KSP preconditioned resid norm 1.074320273869e-002 true resid norm 1.384131369872e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.118408455169e-010 >> 1 KSP preconditioned resid norm 7.118408455169e-010 true resid norm 4.592913592019e-003 ||Ae||/||Ax|| 3.318264213927e-001 >> 16 SNES Function norm 8.126551489387e-003 >> 0 KSP Residual norm 2.817712852237e-009 >> 0 KSP preconditioned resid norm 2.817712852237e-009 true resid norm 8.126551489387e-003 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.239560797627e-016 >> 1 KSP preconditioned resid norm 2.239560797627e-016 true resid norm 1.053395037920e-009 ||Ae||/||Ax|| 1.296238680448e-007 >> 17 SNES Function norm 4.871218193429e-008 >> 0 KSP Residual norm 1.123675925695e-014 >> 0 KSP preconditioned resid norm 1.123675925695e-014 true resid norm 4.871218193429e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.307928330547e-022 >> 1 KSP preconditioned resid norm 5.307928330547e-022 true resid norm 2.893354456541e-015 ||Ae||/||Ax|| 5.939693813026e-008 >> 18 SNES Function norm 5.035279029783e-008 >> STEP 2 (Newton iterations: 18) >> >> diverged reason: -6 >> >> >> Rong >> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Wednesday, May 11, 2011 11:05 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> Ok, the linear solver looks like it is working ok. The likely problem is >>> that Jacobian does not match the function evaluation. >>> >>> Run the same thing but with the additional option -snes_mf_operator >>> >>> Then run with -snes_type test (instead of -snes_mf_operator). >>> >>> Barry >>> >>> On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: >>> >>>> Dear Barry, here is the output using -pc_type >>>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>>> the attached is the same and for clear reference. Thanks again for helps. >>>> >>>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>>> 0 SNES Function norm 7.071067811865e-002 >>>> 0 KSP Residual norm 9.965778978387e-002 >>>> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >>>> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.823187455811e-017 >>>> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >>>> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >>>> 1 SNES Function norm 6.401926523423e-002 >>>> 0 KSP Residual norm 8.969200212486e-002 >>>> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >>>> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.106757475780e-016 >>>> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >>>> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >>>> 2 SNES Function norm 5.849992149767e-002 >>>> 0 KSP Residual norm 8.072279488157e-002 >>>> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >>>> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.268750073799e-017 >>>> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >>>> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >>>> 3 SNES Function norm 5.376618503592e-002 >>>> 0 KSP Residual norm 7.265050969883e-002 >>>> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >>>> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.677655733356e-017 >>>> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >>>> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >>>> 4 SNES Function norm 4.956894354459e-002 >>>> 0 KSP Residual norm 6.538545411661e-002 >>>> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >>>> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 9.557004153175e-017 >>>> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >>>> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >>>> 5 SNES Function norm 4.575418613137e-002 >>>> 0 KSP Residual norm 5.884690496914e-002 >>>> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >>>> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 5.470969262115e-017 >>>> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >>>> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >>>> 6 SNES Function norm 4.223022245585e-002 >>>> 0 KSP Residual norm 5.296221144636e-002 >>>> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >>>> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 8.255198782390e-017 >>>> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >>>> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >>>> 7 SNES Function norm 3.894430065910e-002 >>>> 0 KSP Residual norm 4.766598785088e-002 >>>> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >>>> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 3.322615478395e-017 >>>> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >>>> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >>>> 8 SNES Function norm 3.586683371135e-002 >>>> 0 KSP Residual norm 4.289938708067e-002 >>>> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >>>> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.181358328498e-017 >>>> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >>>> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >>>> 9 SNES Function norm 3.298130202025e-002 >>>> 0 KSP Residual norm 3.860944676473e-002 >>>> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >>>> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 4.635174776374e-017 >>>> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >>>> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >>>> 10 SNES Function norm 3.027806208930e-002 >>>> 0 KSP Residual norm 3.474850078591e-002 >>>> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >>>> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.389914053685e-017 >>>> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >>>> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >>>> 11 SNES Function norm 2.749422924729e-002 >>>> 0 KSP Residual norm 3.081350823297e-002 >>>> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >>>> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.640567497647e-017 >>>> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >>>> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >>>> 12 SNES Function norm 2.437488247885e-002 >>>> 0 KSP Residual norm 2.633007441879e-002 >>>> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >>>> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.772331460094e-017 >>>> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >>>> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >>>> 13 SNES Function norm 2.079664278637e-002 >>>> 0 KSP Residual norm 2.104738289397e-002 >>>> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >>>> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.650632708670e-017 >>>> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >>>> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >>>> 14 SNES Function norm 1.657344626858e-002 >>>> 0 KSP Residual norm 1.454141853505e-002 >>>> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >>>> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.129401160070e-017 >>>> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >>>> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >>>> 15 SNES Function norm 1.484243752612e-002 >>>> 0 KSP Residual norm 5.241948491751e-009 >>>> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >>>> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.729506849025e-024 >>>> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >>>> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >>>> 16 SNES Function norm 2.828002157497e-008 >>>> 0 KSP Residual norm 6.042518362322e-015 >>>> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >>>> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.272441346127e-030 >>>> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >>>> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >>>> 17 SNES Function norm 2.960967020289e-008 >>>> STEP 0 (Newton iterations: 17) >>>> >>>> diverged reason: -6 >>>> >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Wednesday, May 11, 2011 2:54 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor and >>>> send the outputs >>>> >>>> >>>> Barry >>>> >>>> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >>>> >>>>> by the way, the increment size is like that >>>>> for a 100 lengh model, the increment is set to 0.05, >>>>> the engineering strain is around 5%% per load step. >>>>> This is already too small increment size for a large deformation >>>>> analysis. >>>>> a 0.5 increment size leads to both linear search and trust region failed. >>>>> linear search failed for 0.05 while trust region converges with 17 Newton >>>>> iterations each load step. >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Tian(ICT)" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 11:37 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>>> First, thanks again, the issue was gone. >>>>>> >>>>>> I just followed up with some test results. >>>>>> I have tested SNES using one finite element for a geometric large >>>>>> deformation problem. >>>>>> Those are just the very early test results so they may be not telling >>>>>> what happened exactly. >>>>>> For the displacement controlled load, I found that convergence is much >>>>>> slower than that of force loading. >>>>>> Even worse, linear search is so sensitive to the displacement increment >>>>>> and diverged no matter what the increment size was used (too small >>>>>> incremnt also led to diverged soloution (-6 reason), trust region works >>>>>> well in the sense of not sensitive to the displacement increment, but >>>>>> during each load step, it requires around ten to several tens of Newton >>>>>> interations whereas for the force loading case and the almost same >>>>>> amount of deformation, this is normally 3. This is against my >>>>>> expectation. Any hint? >>>>>> >>>>>> Rong >>>>>> >>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>> To: "PETSc users list" >>>>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>> >>>>>> >>>>>> >>>>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>>>> >>>>>>> Dear Barry, Thanks a lot for quick answering. >>>>>>> I checked the development documents and found the new version of >>>>>>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>>>>>> >>>>>>> I followed up with more details. >>>>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>>>>>> solvers. >>>>>>> I used a displacement-controlled load (as this type of loading works >>>>>>> well for all cases). >>>>>>> This is the reason the nonzero prescribed boundary came up. >>>>>>> >>>>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>>>> nonzero prescribed boundary. >>>>>>> In FormFunction, I modified the solution to the known solution(this >>>>>>> should not be necessary as the modified Jacobian and rhs should give >>>>>>> the prescribed solution also) >>>>>> >>>>>> You should not do it this way. See below. >>>>>>> >>>>>>> Now I found another issue, no matter if I prescried the solution or not >>>>>>> in FormFunction, >>>>>>> SNES solver always call FormFunction and never call FormJacobian. >>>>>> >>>>>> The only reason it would not call FormJacobian is if decided that the >>>>>> residual norm was small enough before any Newton steps; for example if >>>>>> the FormFunction() computed exactly the zero function initially. When >>>>>> you run with -snes_monitor -ksp_monitor what does it print for residual >>>>>> norms. >>>>>> >>>>>>> Of course the solver finally diverged or converged to a zero solution. >>>>>>> >>>>>>> So my quick follow up question is How a displacement-controled load is >>>>>>> done corrently in Petsc 2.3.3? >>>>>> >>>>>> To do it in 2.3.3 simply have for those components of F() the formula >>>>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for >>>>>> those rows >>>>>> >>>>>> We strongly urge you to upgrade to the latest PETSc before doing >>>>>> anything further. >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>>> >>>>>>> Rong >>>>>>> >>>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>>> To: "PETSc users list" >>>>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>>> >>>>>>> >>>>>>> >>>>>>> In petsc-dev >>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>>>> modified the calling sequence for MatZeroRows() so that it can >>>>>>> automatically adjust the appropriate right hand side values for the >>>>>>> zeroed rows to support zero or non-zero prescribed boundary conditions >>>>>>> easily. >>>>>>> >>>>>>> Barry >>>>>>> >>>>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>>>> >>>>>>>> Dear all, >>>>>>>> >>>>>>>> I got this question long ago and searched the prior posting but did >>>>>>>> not find the solution. >>>>>>>> The question is about nonzero prescribed boundary condition. >>>>>>>> My understanding is that MatZeroRows() works only for zero prescribed >>>>>>>> value, not non-zero value. >>>>>>>> For the non-zero values, we have to remove the rows associated with >>>>>>>> the boundary, but this >>>>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>>>> also be removed. >>>>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>>>> prescribed boundary and if so how to do it simply? >>>>>>>> >>>>>>>> Rong >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >> > > From bsmith at mcs.anl.gov Wed May 11 14:03:46 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 11 May 2011 14:03:46 -0500 Subject: [petsc-users] Error of petsc builde or the conflic between Petsc and intel-window Microsoft Visual Stud compiler In-Reply-To: <9393A2DE2F16CD4ABD3B95167C7DFC1202A288@mbs02.uit.no> References: <9393A2DE2F16CD4ABD3B95167C7DFC1202A288@mbs02.uit.no> Message-ID: This is a very strange error. To pointers are coming out to be 0 when there is no way they should ever be. First cd to src/vec/vec/examples/tutorials ; make ex1; ./ex1 does that run ok? Then cd src/snes/examples/tutorials; make ex19; ./ex19 does that crash (likely it will) Then you need to run either ex1 (if it crashed) or ex19 in the debugger to see how the pointers in the PetscMemcpy() are ending up being zero. One thought is that some incompatible system library is getting linked in. Are you sure the install of Developers studio is correct and there are not any old compiler libraries around? That is does a simple developers studio code compile and run correctly? Barry On May 10, 2011, at 6:57 AM, Nguyen Hung Thanh wrote: > Dear Petsc developers/users, > It took a while for me to install Petsc, configure and used this library on Intel ?Microsoft Visual platform. And, unfortunately I have not reached the goal yet. I am sorry for that I have again ask you some questions about Petsc configure/ running: > > 1) I used the configure-options is > > Hung at TRD-K-ST-008-PC ~/petsc-3.1-p7 > $ ./config/configure.py --with-cc='win32fe icl' --with-cxx='win32fe icl' --with > -clanguage=cxx --with-fc=0 --with-mpi-include=/home/Hung/IntelMPI/MPI/4.0.1.0 > 07/ia32/include --with-mpi-lib=/home/Hung/IntelMPI/MPI/4.0.1.007/ia32/lib/imp > i.lib --with-blas-lapack-dir=/home/Hung/IntelMKL/ia32/lib > NB: impi > The configure is seem ok. > > 2) And, check the Petcs libraries working :$ make PETSC_DIR=/home/Hung/petsc-3.1-p7 PETSC_ARCH=cygwin-cxx-debug test > > Hung at TRD-K-ST-008-PC ~/petsc-3.1-p7 > $ make PETSC_DIR=/home/Hung/petsc-3.1-p7 PETSC_ARCH=cygwin-cxx-debug test > Running test examples to verify correct installation > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process > See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html > [0]PETSC ERROR: --------------------- Error Message ---------------------------- > -------- > [0]PETSC ERROR: Arguments are incompatible! > [0]PETSC ERROR: Memory regions overlap: either use PetscMemmov() > or make sure your copy regions and lengths are correct. > Length (bytes) 64 first address 0 second address 0! > [0]PETSC ERROR: ---------------------------------------------------------------- > -------- > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 20 > 10 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ---------------------------------------------------------------- > -------- > [0]PETSC ERROR: C:\cygwin\home\Hung\petsc-3.1-p7\src\snes\examples\tutorials\ex1 > 9.exe on a cygwin-cx named TRD-K-ST-008-PC by Hung Tue May 10 13:35:26 2011 > [0]PETSC ERROR: Libraries linked from /home/Hung/petsc-3.1-p7/cygwin-cxx-debug/l > ib > [0]PETSC ERROR: Configure run at Tue May 10 11:28:56 2011 > [0]PETSC ERROR: Configure options --with-cc="win32fe icl" --with-cxx="win32fe ic > l" --with-clanguage=cxx --with-fc=0 --with-mpi-include=/home/Hung/IntelMPI/MPI/4 > .0.1.007/ia32/include --with-mpi-lib=/home/Hung/IntelMPI/MPI/4.0.1.007/ia32/lib/ > impi.lib --with-blas-lapack-dir=/home/Hung/IntelMKL/ia32/lib --useThreads=0 > [0]PETSC ERROR: ---------------------------------------------------------------- > -------- > [0]PETSC ERROR: User provided function() line 1755 in src/vec/is/impls/block/C:\ > cygwin\home\Hung\PETSC-~1.1-P\include\petscsys.h > [0]PETSC ERROR: ISCreateBlock() line 296 in src/vec/is/impls/block/C:\cygwin\hom > e\Hung\PETSC-~1.1-P\src\vec\is\impls\block\block.c > [0]PETSC ERROR: DACreate_2D() line 1463 in src/dm/da/src/C:\cygwin\home\Hung\PET > SC-~1.1-P\src\dm\da\src\da2.c > [0]PETSC ERROR: DASetType() line 48 in src/dm/da/src/C:\cygwin\home\Hung\PETSC-~ > 1.1-P\src\dm\da\src\dareg.c > [0]PETSC ERROR: DASetTypeFromOptions_Private() line 65 in src/dm/da/src/C:\cygwi > n\home\Hung\PETSC-~1.1-P\src\dm\da\src\dacreate.c > [0]PETSC ERROR: DASetFromOptions() line 131 in src/dm/da/src/C:\cygwin\home\Hung > \PETSC-~1.1-P\src\dm\da\src\dacreate.c > [0]PETSC ERROR: DACreate2d() line 1847 in src/dm/da/src/C:\cygwin\home\Hung\PETS > C-~1.1-P\src\dm\da\src\da2.c > [0]PETSC ERROR: main() line 107 in src/snes/examples/tutorials/C:\cygwin\home\Hu > ng\PETSC-~1.1-P\src\snes\examples\TUTORI~1\ex19.c > application called MPI_Abort(MPI_COMM_WORLD, 75) - process 0 > job aborted: > rank: node: exit code[: error message] > 0: TRD-K-ST-008-PC.petrell.local: 75: process 0 exited without calling finalize > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI process > es..... > See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html > Completed test examples > > Please, tell me how I can solve the problem, > Regards > Hung T Nguyen > Numerical analys > Physis Derpart. > UiT?, Troms? Norway > From rongtian at ncic.ac.cn Wed May 11 23:35:18 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Thu, 12 May 2011 12:35:18 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop><1302D684D3644298A3F2528BDEE08857@rti458laptop><32DB8456522B4FABB725933C113D66D1@rti458laptop> <03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov> Message-ID: <71C9B313096748028405BE22EBAF85CD@rti458laptop> Thank you very much for accurate hints. There was a mistake in computing functions, which led to wrong jacobian. Now both function and jacobian were verfied against the other code and they are corrent now. Two tests were attached below: [1] Without any option, SNES converges never calling FormJacobian, and the solution seems correct. What is the reason for that the SNES solver never calls FormJacobian (mf option was not used)? 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | STEP 0 (Newton iterations: 3) 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | Form Function ... 12:16:37 | info | STEP 1 (Newton iterations: 3) ******************************************** ******************************************** ******************************************** [2] Run with -pc_type none -ksp_monitor_true_residual -snes_monitor -snes_mf ******************************************** ******************************************** ******************************************** again, FormJacobian was never be called, the outputs are as follow. both cases, SNES seems much slower than a hand-coded solver using the arc-length search. Any further hint? atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067811865e+001 0 KSP preconditioned resid norm 7.071067811865e+001 true resid norm 7.071067811865e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.198922749247e+001 true resid norm 6.198922749247e+001 ||Ae||/||Ax|| 8.766600624088e-001 2 KSP preconditioned resid norm 5.528279492085e+001 true resid norm 5.528279475119e+001 ||Ae||/||Ax|| 7.818167810303e-001 3 KSP preconditioned resid norm 5.297780256406e+001 true resid norm 5.297780279950e+001 ||Ae||/||Ax|| 7.492192722378e-001 4 KSP preconditioned resid norm 5.114021551683e+001 true resid norm 5.114021331426e+001 ||Ae||/||Ax|| 7.232318325168e-001 5 KSP preconditioned resid norm 5.083804009339e+001 true resid norm 5.083804105448e+001 ||Ae||/||Ax|| 7.189584714373e-001 6 KSP preconditioned resid norm 5.046749215177e+001 true resid norm 5.046749115857e+001 ||Ae||/||Ax|| 7.137181045540e-001 7 KSP preconditioned resid norm 5.017876322892e+001 true resid norm 5.017876245111e+001 ||Ae||/||Ax|| 7.096348640145e-001 8 KSP preconditioned resid norm 5.014607662584e+001 true resid norm 5.014607460521e+001 ||Ae||/||Ax|| 7.091725880646e-001 9 KSP preconditioned resid norm 5.013735265505e+001 true resid norm 5.013735450946e+001 ||Ae||/||Ax|| 7.090492672879e-001 10 KSP preconditioned resid norm 5.013129184370e+001 true resid norm 5.013129107602e+001 ||Ae||/||Ax|| 7.089635173898e-001 11 KSP preconditioned resid norm 5.010794870348e+001 true resid norm 5.010794800982e+001 ||Ae||/||Ax|| 7.086333965817e-001 12 KSP preconditioned resid norm 4.997965989272e+001 true resid norm 4.997967683972e+001 ||Ae||/||Ax|| 7.068193682975e-001 13 KSP preconditioned resid norm 3.925485685703e+001 true resid norm 3.925223649295e+001 ||Ae||/||Ax|| 5.551104520181e-001 14 KSP preconditioned resid norm 3.766533768027e+001 true resid norm 3.766438346984e+001 ||Ae||/||Ax|| 5.326548192147e-001 15 KSP preconditioned resid norm 1.679441681862e+001 true resid norm 1.679359300016e+001 ||Ae||/||Ax|| 2.374972698180e-001 16 KSP preconditioned resid norm 1.250214314546e-009 true resid norm 3.622974465129e-001 ||Ae||/||Ax|| 5.123659624716e-003 1 SNES Function norm 5.010346201281e+000 0 KSP preconditioned resid norm 5.010346201281e+000 true resid norm 5.010346201281e+000 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 3.125067154800e+000 true resid norm 3.125067154800e+000 ||Ae||/||Ax|| 6.237227986364e-001 2 KSP preconditioned resid norm 1.790676902005e+000 true resid norm 1.790676998514e+000 ||Ae||/||Ax|| 3.573958617982e-001 3 KSP preconditioned resid norm 9.791340982396e-001 true resid norm 9.791340674390e-001 ||Ae||/||Ax|| 1.954224375131e-001 4 KSP preconditioned resid norm 4.169136961659e-001 true resid norm 4.169135681315e-001 ||Ae||/||Ax|| 8.321053104573e-002 5 KSP preconditioned resid norm 1.952800046013e-001 true resid norm 1.952800146848e-001 ||Ae||/||Ax|| 3.897535356635e-002 6 KSP preconditioned resid norm 1.371162184665e-001 true resid norm 1.371160776663e-001 ||Ae||/||Ax|| 2.736658748875e-002 7 KSP preconditioned resid norm 4.752749582423e-002 true resid norm 4.752714319348e-002 ||Ae||/||Ax|| 9.485800238980e-003 8 KSP preconditioned resid norm 3.679436716634e-002 true resid norm 3.679416671320e-002 ||Ae||/||Ax|| 7.343637592108e-003 9 KSP preconditioned resid norm 3.174486861332e-002 true resid norm 3.174525181575e-002 ||Ae||/||Ax|| 6.335939781494e-003 10 KSP preconditioned resid norm 2.860943594055e-002 true resid norm 2.860914164631e-002 ||Ae||/||Ax|| 5.710012940622e-003 11 KSP preconditioned resid norm 2.502632777351e-002 true resid norm 2.502627096448e-002 ||Ae||/||Ax|| 4.994918506446e-003 12 KSP preconditioned resid norm 2.479081072702e-002 true resid norm 2.479081050011e-002 ||Ae||/||Ax|| 4.947923657207e-003 13 KSP preconditioned resid norm 2.453552472271e-002 true resid norm 2.453550748659e-002 ||Ae||/||Ax|| 4.896968492978e-003 14 KSP preconditioned resid norm 2.450340222151e-002 true resid norm 2.450340723637e-002 ||Ae||/||Ax|| 4.890561700129e-003 15 KSP preconditioned resid norm 2.446841894425e-002 true resid norm 2.446842016638e-002 ||Ae||/||Ax|| 4.883578735563e-003 16 KSP preconditioned resid norm 1.185777680895e-012 true resid norm 7.269638622157e-004 ||Ae||/||Ax|| 1.450925411162e-004 2 SNES Function norm 1.285031516032e-004 0 KSP preconditioned resid norm 1.285031516032e-004 true resid norm 1.285031516032e-004 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 7.358333039693e-005 true resid norm 7.358333039693e-005 ||Ae||/||Ax|| 5.726188772719e-001 2 KSP preconditioned resid norm 2.793571629313e-005 true resid norm 2.793571561993e-005 ||Ae||/||Ax|| 2.173932333285e-001 3 KSP preconditioned resid norm 1.098998624012e-005 true resid norm 1.098998762581e-005 ||Ae||/||Ax|| 8.552309798399e-002 4 KSP preconditioned resid norm 6.860729498044e-006 true resid norm 6.860733291260e-006 ||Ae||/||Ax|| 5.338961111588e-002 5 KSP preconditioned resid norm 3.936073017539e-006 true resid norm 3.936066932777e-006 ||Ae||/||Ax|| 3.063011983496e-002 6 KSP preconditioned resid norm 3.270321923106e-006 true resid norm 3.270323858459e-006 ||Ae||/||Ax|| 2.544936694282e-002 7 KSP preconditioned resid norm 2.872966498200e-006 true resid norm 2.872962085195e-006 ||Ae||/||Ax|| 2.235713326367e-002 8 KSP preconditioned resid norm 2.685995411641e-006 true resid norm 2.686000372103e-006 ||Ae||/||Ax|| 2.090221398146e-002 9 KSP preconditioned resid norm 2.163968160949e-006 true resid norm 2.163969192640e-006 ||Ae||/||Ax|| 1.683981416520e-002 10 KSP preconditioned resid norm 1.623010286027e-006 true resid norm 1.623006420289e-006 ||Ae||/||Ax|| 1.263009039109e-002 11 KSP preconditioned resid norm 1.296917697338e-006 true resid norm 1.296928077393e-006 ||Ae||/||Ax|| 1.009257797348e-002 12 KSP preconditioned resid norm 9.434402087990e-007 true resid norm 9.434436935582e-007 ||Ae||/||Ax|| 7.341794203393e-003 13 KSP preconditioned resid norm 3.488471072668e-007 true resid norm 3.488451352886e-007 ||Ae||/||Ax|| 2.714681554004e-003 14 KSP preconditioned resid norm 2.692414749742e-007 true resid norm 2.692386518112e-007 ||Ae||/||Ax|| 2.095191039692e-003 15 KSP preconditioned resid norm 3.070780908139e-008 true resid norm 3.070820208411e-008 ||Ae||/||Ax|| 2.389684743214e-004 16 KSP preconditioned resid norm 7.108045702394e-019 true resid norm 1.174370495958e-009 ||Ae||/||Ax|| 9.138845867256e-006 3 SNES Function norm 4.267299670732e-007 STEP 0 (Newton iterations: 3) 0 SNES Function norm 7.071067801331e+001 0 KSP preconditioned resid norm 7.071067801331e+001 true resid norm 7.071067801331e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.198533980359e+001 true resid norm 6.198533980359e+001 ||Ae||/||Ax|| 8.766050834913e-001 2 KSP preconditioned resid norm 5.527994274677e+001 true resid norm 5.527994343828e+001 ||Ae||/||Ax|| 7.817764585410e-001 3 KSP preconditioned resid norm 5.297611644586e+001 true resid norm 5.297611706439e+001 ||Ae||/||Ax|| 7.491954334594e-001 4 KSP preconditioned resid norm 5.113922798981e+001 true resid norm 5.113922857252e+001 ||Ae||/||Ax|| 7.232179072430e-001 5 KSP preconditioned resid norm 5.083735189265e+001 true resid norm 5.083735192008e+001 ||Ae||/||Ax|| 7.189487266762e-001 6 KSP preconditioned resid norm 5.046773176595e+001 true resid norm 5.046773017799e+001 ||Ae||/||Ax|| 7.137214858622e-001 7 KSP preconditioned resid norm 5.017895995906e+001 true resid norm 5.017896133115e+001 ||Ae||/||Ax|| 7.096376776602e-001 8 KSP preconditioned resid norm 5.014624418384e+001 true resid norm 5.014624362257e+001 ||Ae||/||Ax|| 7.091749793875e-001 9 KSP preconditioned resid norm 5.013751425980e+001 true resid norm 5.013751438456e+001 ||Ae||/||Ax|| 7.090515293195e-001 10 KSP preconditioned resid norm 5.013145929324e+001 true resid norm 5.013145910954e+001 ||Ae||/||Ax|| 7.089658947988e-001 11 KSP preconditioned resid norm 5.010818111716e+001 true resid norm 5.010818054884e+001 ||Ae||/||Ax|| 7.086366862358e-001 12 KSP preconditioned resid norm 4.998040225128e+001 true resid norm 4.998039927752e+001 ||Ae||/||Ax|| 7.068295861639e-001 13 KSP preconditioned resid norm 3.925980557222e+001 true resid norm 3.926954729376e+001 ||Ae||/||Ax|| 5.553552645382e-001 14 KSP preconditioned resid norm 3.767160996011e+001 true resid norm 3.766320856308e+001 ||Ae||/||Ax|| 5.326382043175e-001 15 KSP preconditioned resid norm 1.689041692217e+001 true resid norm 1.690596162049e+001 ||Ae||/||Ax|| 2.390864024428e-001 16 KSP preconditioned resid norm 7.494696199747e-010 true resid norm 7.321692289701e-001 ||Ae||/||Ax|| 1.035443655104e-002 1 SNES Function norm 5.065297893048e+000 0 KSP preconditioned resid norm 5.065297893048e+000 true resid norm 5.065297893048e+000 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 3.152895394153e+000 true resid norm 3.152895394153e+000 ||Ae||/||Ax|| 6.224501422672e-001 2 KSP preconditioned resid norm 1.802574999082e+000 true resid norm 1.802575011564e+000 ||Ae||/||Ax|| 3.558675224291e-001 3 KSP preconditioned resid norm 9.797025431577e-001 true resid norm 9.797029717256e-001 ||Ae||/||Ax|| 1.934146801258e-001 4 KSP preconditioned resid norm 4.175614713454e-001 true resid norm 4.175593934986e-001 ||Ae||/||Ax|| 8.243530831064e-002 5 KSP preconditioned resid norm 1.966470605735e-001 true resid norm 1.966475028867e-001 ||Ae||/||Ax|| 3.882249515011e-002 6 KSP preconditioned resid norm 1.381279214633e-001 true resid norm 1.381273768772e-001 ||Ae||/||Ax|| 2.726934916638e-002 7 KSP preconditioned resid norm 4.766343518370e-002 true resid norm 4.766398522151e-002 ||Ae||/||Ax|| 9.409907616080e-003 8 KSP preconditioned resid norm 3.681767230431e-002 true resid norm 3.681839498297e-002 ||Ae||/||Ax|| 7.268752156414e-003 9 KSP preconditioned resid norm 3.154913051307e-002 true resid norm 3.154870156685e-002 ||Ae||/||Ax|| 6.228400033520e-003 10 KSP preconditioned resid norm 2.863814386382e-002 true resid norm 2.863836535365e-002 ||Ae||/||Ax|| 5.653836350465e-003 11 KSP preconditioned resid norm 2.516240323861e-002 true resid norm 2.516228989658e-002 ||Ae||/||Ax|| 4.967583432973e-003 12 KSP preconditioned resid norm 2.499694019590e-002 true resid norm 2.499694632312e-002 ||Ae||/||Ax|| 4.934941014512e-003 13 KSP preconditioned resid norm 2.474701691610e-002 true resid norm 2.474699653509e-002 ||Ae||/||Ax|| 4.885595488679e-003 14 KSP preconditioned resid norm 2.470500041453e-002 true resid norm 2.470500324132e-002 ||Ae||/||Ax|| 4.877305098922e-003 15 KSP preconditioned resid norm 2.466252351205e-002 true resid norm 2.466252750272e-002 ||Ae||/||Ax|| 4.868919464059e-003 16 KSP preconditioned resid norm 7.240664411016e-013 true resid norm 8.621971002033e-004 ||Ae||/||Ax|| 1.702164647388e-004 2 SNES Function norm 1.715637353858e-004 0 KSP preconditioned resid norm 1.715637353858e-004 true resid norm 1.715637353858e-004 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 5.133805201182e-005 true resid norm 5.133805201182e-005 ||Ae||/||Ax|| 2.992360354965e-001 2 KSP preconditioned resid norm 3.105488497537e-005 true resid norm 3.105488438607e-005 ||Ae||/||Ax|| 1.810107731464e-001 3 KSP preconditioned resid norm 1.395061713608e-005 true resid norm 1.395061820355e-005 ||Ae||/||Ax|| 8.131449325334e-002 4 KSP preconditioned resid norm 7.751134073702e-006 true resid norm 7.751135728902e-006 ||Ae||/||Ax|| 4.517933648082e-002 5 KSP preconditioned resid norm 4.654853308327e-006 true resid norm 4.654854773883e-006 ||Ae||/||Ax|| 2.713192717223e-002 6 KSP preconditioned resid norm 4.251658354500e-006 true resid norm 4.251660009778e-006 ||Ae||/||Ax|| 2.478181067938e-002 7 KSP preconditioned resid norm 3.555184986154e-006 true resid norm 3.555185020946e-006 ||Ae||/||Ax|| 2.072224070519e-002 8 KSP preconditioned resid norm 3.215587707994e-006 true resid norm 3.215587246709e-006 ||Ae||/||Ax|| 1.874281438020e-002 9 KSP preconditioned resid norm 2.634842773217e-006 true resid norm 2.634843527693e-006 ||Ae||/||Ax|| 1.535781161309e-002 10 KSP preconditioned resid norm 2.036964505740e-006 true resid norm 2.036965395860e-006 ||Ae||/||Ax|| 1.187293684927e-002 11 KSP preconditioned resid norm 1.746186350035e-006 true resid norm 1.746185026306e-006 ||Ae||/||Ax|| 1.017805436784e-002 12 KSP preconditioned resid norm 1.302769828754e-006 true resid norm 1.302771152061e-006 ||Ae||/||Ax|| 7.593511234361e-003 13 KSP preconditioned resid norm 7.177208286434e-007 true resid norm 7.177260379251e-007 ||Ae||/||Ax|| 4.183436763667e-003 14 KSP preconditioned resid norm 5.550901925209e-007 true resid norm 5.550891982416e-007 ||Ae||/||Ax|| 3.235469296547e-003 15 KSP preconditioned resid norm 3.391040385332e-008 true resid norm 3.391065128579e-008 ||Ae||/||Ax|| 1.976562891309e-004 16 KSP preconditioned resid norm 3.363862755900e-018 true resid norm 9.196863532382e-010 ||Ae||/||Ax|| 5.360610452844e-006 3 SNES Function norm 2.731179168679e-007 STEP 1 (Newton iterations: 3) ----Rong ----- Original Message ----- From: "Barry Smith" To: "PETSc users list" Sent: Thursday, May 12, 2011 12:15 AM Subject: Re: [petsc-users] nonzero prescribed boundary condition Ok, Run with -pc_type none -ksp_monitor_true_residual -snes_monitor -snes_mf Send the output Barry On May 11, 2011, at 8:44 AM, Tian(ICT) wrote: > C:\cygwin\home\rabbit\pGFEM-test>pgfem -pc_type > lu -ksp_monitor_true_residual -s > nes_monitor > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm > 7.07106781 > 1865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm > 8.84729888 > 5656e-011 ||Ae||/||Ax|| 1.251197007446e-009 > 1 SNES Function norm 6.401926523423e-002 > 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm > 6.40192652 > 3423e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm > 6.21183006 > 7439e-011 ||Ae||/||Ax|| 9.703063671087e-010 > 2 SNES Function norm 5.849992149767e-002 > 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm > 5.84999214 > 9767e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm > 3.80243103 > 6387e-011 ||Ae||/||Ax|| 6.499890835816e-010 > 3 SNES Function norm 5.376618503592e-002 > 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm > 5.37661850 > 3592e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm > 8.12039778 > 8686e-011 ||Ae||/||Ax|| 1.510316899602e-009 > 4 SNES Function norm 4.956894354459e-002 > 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm > 4.95689435 > 4459e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm > 2.94425080 > 2029e-011 ||Ae||/||Ax|| 5.939708598754e-010 > 5 SNES Function norm 4.575418613137e-002 > 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm > 4.57541861 > 3137e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm > 3.65900316 > 6095e-011 ||Ae||/||Ax|| 7.997089393284e-010 > 6 SNES Function norm 4.223022245585e-002 > 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm > 4.22302224 > 5585e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm > 1.95554565 > 8933e-011 ||Ae||/||Ax|| 4.630678090739e-010 > 7 SNES Function norm 3.894430065910e-002 > 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm > 3.89443006 > 5910e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm > 3.48532814 > 8673e-011 ||Ae||/||Ax|| 8.949520442496e-010 > 8 SNES Function norm 3.586683371135e-002 > 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm > 3.58668337 > 1135e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm > 3.24690281 > 8086e-011 ||Ae||/||Ax|| 9.052660862724e-010 > 9 SNES Function norm 3.298130202025e-002 > 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm > 3.29813020 > 2025e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm > 1.49751684 > 2272e-011 ||Ae||/||Ax|| 4.540502498513e-010 > 10 SNES Function norm 3.027806208930e-002 > 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm > 3.02780620 > 8930e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm > 6.00744088 > 8596e-011 ||Ae||/||Ax|| 1.984090286517e-009 > 11 SNES Function norm 2.749422924729e-002 > 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm > 2.74942292 > 4729e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm > 1.28163829 > 5853e-011 ||Ae||/||Ax|| 4.661481085089e-010 > 12 SNES Function norm 2.437488247885e-002 > 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm > 2.43748824 > 7885e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm > 1.91821249 > 6143e-011 ||Ae||/||Ax|| 7.869627670236e-010 > 13 SNES Function norm 2.079664278637e-002 > 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm > 2.07966427 > 8637e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm > 2.31637196 > 7362e-011 ||Ae||/||Ax|| 1.113820144509e-009 > 14 SNES Function norm 1.657344626858e-002 > 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm > 1.65734462 > 6858e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm > 7.88549932 > 7559e-012 ||Ae||/||Ax|| 4.757911661686e-010 > 15 SNES Function norm 1.484243752612e-002 > 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm > 1.48424375 > 2612e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm > 6.38667785 > 1085e-018 ||Ae||/||Ax|| 4.302984492839e-016 > 16 SNES Function norm 2.828002157497e-008 > 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm > 2.82800215 > 7497e-008 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm > 1.11285769 > 8032e-023 ||Ae||/||Ax|| 3.935137372797e-016 > 17 SNES Function norm 2.960967020289e-008 > STEP 0 (Newton iterations: 17) > > diverged reason: -6 > > C:\cygwin\home\rabbit\pGFEM-test> > > > ==================HOWEVER================= > -snes_type tr seems work > > C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_type tr > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > STEP 0 (Newton iterations: 12) > > STEP 1 (Newton iterations: 11) > > STEP 2 (Newton iterations: 11) > ... > > STEP 70 (Newton iterations: 14) > > ^C > C:\cygwin\home\rabbit\pGFEM-test> > > ----- Original Message ----- From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 9:02 PM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > >>> Testing hand-coded Jacobian, if the ratio is >>> O(1.e-8), the hand-coded Jacobian is probably correct. >>> Run with -snes_test_display to show difference >>> of hand-coded and finite difference Jacobian. >>> Norm of matrix ratio 9.05702e-008 difference 3.94971 >>> Norm of matrix ratio 3.6015e-007 difference 15.7059 >>> Norm of matrix ratio 3.73941e-007 difference 16.3073 > > Based on this it appears the Jacobian at the initial point is correct but > > 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm > 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm > 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 > > this indicates that the linear solver is not actually solving the linear > system. One would expect a large drop in the "true resid norm" and one > does not see that. This could be caused by > > 1) the computed Jacobian is wrong or > > 2) the Jacobian is singular and that is not being taken into account in > the linear solver. > > Please rerun with the options -pc_type > lu -ksp_monitor_true_residual -snes_monitor BUT NOT THE > OPTION -snes_mf_operator and send the same output. > > Barry > > > On May 11, 2011, at 1:46 AM, Tian(ICT) wrote: > >> Pls ignore my previous email. >> >> The followings are the test results: >> the previous error in using -snes_mf_opertor was due to my own code. >> After modification, i.e. using different A and B and assemble both >> and in FormJacobian the native jacobian was assembled into B. >> Then with -snes_mf_operator led to the results shown below. >> The first three steps converged but with a diverged reason -6 at step 4. >> >> >> With additional -snes_type test, the results were >> >>> "-pc_type >>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type >>> test" >>> >>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>> Testing hand-coded Jacobian, if the ratio is >>> O(1.e-8), the hand-coded Jacobian is probably correct. >>> Run with -snes_test_display to show difference >>> of hand-coded and finite difference Jacobian. >>> Norm of matrix ratio 9.05702e-008 difference 3.94971 >>> Norm of matrix ratio 3.6015e-007 difference 15.7059 >>> Norm of matrix ratio 3.73941e-007 difference 16.3073 >>> [0]PETSC ERROR: SNESSolve() line 1871 in >>> src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c >>> [0]PETSC ERROR: User provided function() line 572 in >>> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp >>> [0]PETSC ERROR: User provided function() line 40 in >>> unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp >>> >>> >>> "-pc_type >>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> 0 SNES Function norm 7.071067811865e-002 >> 0 KSP Residual norm 9.965778978387e-002 >> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.102368266281e-009 >> 1 KSP preconditioned resid norm 9.102368266281e-009 true resid norm >> 5.414197277222e-002 ||Ae||/||Ax|| 7.656831218811e-001 >> 1 SNES Function norm 6.401926196865e-002 >> 0 KSP Residual norm 8.969200166800e-002 >> 0 KSP preconditioned resid norm 8.969200166800e-002 true resid norm >> 6.401926196865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.492849351675e-009 >> 1 KSP preconditioned resid norm 8.492849351675e-009 true resid norm >> 1.400035213572e-002 ||Ae||/||Ax|| 2.186896834670e-001 >> 2 SNES Function norm 5.849991037388e-002 >> 0 KSP Residual norm 8.072279482667e-002 >> 0 KSP preconditioned resid norm 8.072279482667e-002 true resid norm >> 5.849991037388e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.610510067555e-009 >> 1 KSP preconditioned resid norm 3.610510067555e-009 true resid norm >> 4.757549946973e-003 ||Ae||/||Ax|| 8.132576471599e-002 >> 3 SNES Function norm 5.376617142827e-002 >> 0 KSP Residual norm 7.265050986128e-002 >> 0 KSP preconditioned resid norm 7.265050986128e-002 true resid norm >> 5.376617142827e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.941165647221e-009 >> 1 KSP preconditioned resid norm 6.941165647221e-009 true resid norm >> 3.935270526562e-002 ||Ae||/||Ax|| 7.319231446138e-001 >> 4 SNES Function norm 4.956888929193e-002 >> 0 KSP Residual norm 6.538545408372e-002 >> 0 KSP preconditioned resid norm 6.538545408372e-002 true resid norm >> 4.956888929193e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.457970649220e-009 >> 1 KSP preconditioned resid norm 3.457970649220e-009 true resid norm >> 2.037700110050e-002 ||Ae||/||Ax|| 4.110844804388e-001 >> 5 SNES Function norm 4.575415001780e-002 >> 0 KSP Residual norm 5.884690527647e-002 >> 0 KSP preconditioned resid norm 5.884690527647e-002 true resid norm >> 4.575415001780e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.832662187541e-009 >> 1 KSP preconditioned resid norm 2.832662187541e-009 true resid norm >> 1.662420124938e-002 ||Ae||/||Ax|| 3.633375604818e-001 >> 6 SNES Function norm 4.223018337199e-002 >> 0 KSP Residual norm 5.296221162333e-002 >> 0 KSP preconditioned resid norm 5.296221162333e-002 true resid norm >> 4.223018337199e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.091667005099e-009 >> 1 KSP preconditioned resid norm 3.091667005099e-009 true resid norm >> 2.235399010718e-002 ||Ae||/||Ax|| 5.293367994705e-001 >> 7 SNES Function norm 3.894426990170e-002 >> 0 KSP Residual norm 4.766598828506e-002 >> 0 KSP preconditioned resid norm 4.766598828506e-002 true resid norm >> 3.894426990170e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.223811830585e-009 >> 1 KSP preconditioned resid norm 4.223811830585e-009 true resid norm >> 2.831156780040e-002 ||Ae||/||Ax|| 7.269764684732e-001 >> 8 SNES Function norm 3.586679674893e-002 >> 0 KSP Residual norm 4.289938722959e-002 >> 0 KSP preconditioned resid norm 4.289938722959e-002 true resid norm >> 3.586679674893e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.504535460429e-009 >> 1 KSP preconditioned resid norm 3.504535460429e-009 true resid norm >> 2.254598873838e-002 ||Ae||/||Ax|| 6.286033541328e-001 >> 9 SNES Function norm 3.298126871313e-002 >> 0 KSP Residual norm 3.860944657267e-002 >> 0 KSP preconditioned resid norm 3.860944657267e-002 true resid norm >> 3.298126871313e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.439084585574e-009 >> 1 KSP preconditioned resid norm 4.439084585574e-009 true resid norm >> 6.936792391193e-003 ||Ae||/||Ax|| 2.103252137305e-001 >> 10 SNES Function norm 3.027806037514e-002 >> 0 KSP Residual norm 3.474850031738e-002 >> 0 KSP preconditioned resid norm 3.474850031738e-002 true resid norm >> 3.027806037514e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.476854671229e-009 >> 1 KSP preconditioned resid norm 3.476854671229e-009 true resid norm >> 1.935717696630e-002 ||Ae||/||Ax|| 6.393136391984e-001 >> 11 SNES Function norm 2.775064646763e-002 >> 0 KSP Residual norm 3.127364888698e-002 >> 0 KSP preconditioned resid norm 3.127364888698e-002 true resid norm >> 2.775064646763e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.380857715198e-009 >> 1 KSP preconditioned resid norm 2.380857715198e-009 true resid norm >> 1.646856223644e-002 ||Ae||/||Ax|| 5.934478771746e-001 >> 12 SNES Function norm 2.430399371900e-002 >> 0 KSP Residual norm 2.602654480862e-002 >> 0 KSP preconditioned resid norm 2.602654480862e-002 true resid norm >> 2.430399371900e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.556142726483e-009 >> 1 KSP preconditioned resid norm 2.556142726483e-009 true resid norm >> 1.794651743617e-002 ||Ae||/||Ax|| 7.384184526900e-001 >> 13 SNES Function norm 2.029256664631e-002 >> 0 KSP Residual norm 1.954645630691e-002 >> 0 KSP preconditioned resid norm 1.954645630691e-002 true resid norm >> 2.029256664631e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.913585412092e-010 >> 1 KSP preconditioned resid norm 3.913585412092e-010 true resid norm >> 1.903675728342e-003 ||Ae||/||Ax|| 9.381148089946e-002 >> 14 SNES Function norm 1.566638821565e-002 >> 0 KSP Residual norm 1.231404128086e-002 >> 0 KSP preconditioned resid norm 1.231404128086e-002 true resid norm >> 1.566638821565e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.258598971379e-010 >> 1 KSP preconditioned resid norm 7.258598971379e-010 true resid norm >> 1.221317129610e-003 ||Ae||/||Ax|| 7.795779810881e-002 >> 15 SNES Function norm 1.064422587384e-002 >> 0 KSP Residual norm 3.723145759360e-009 >> 0 KSP preconditioned resid norm 3.723145759360e-009 true resid norm >> 1.064422587384e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.546825086207e-016 >> 1 KSP preconditioned resid norm 5.546825086207e-016 true resid norm >> 2.984647735625e-009 ||Ae||/||Ax|| 2.804006389003e-007 >> 16 SNES Function norm 9.689226075617e-008 >> 0 KSP Residual norm 1.643080767133e-014 >> 0 KSP preconditioned resid norm 1.643080767133e-014 true resid norm >> 9.689226075617e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.291965935374e-022 >> 1 KSP preconditioned resid norm 7.291965935374e-022 true resid norm >> 5.001850432973e-015 ||Ae||/||Ax|| 5.162280654758e-008 >> 17 SNES Function norm 3.747290584108e-008 >> STEP 0 (Newton iterations: 17) >> >> 0 SNES Function norm 7.071067811866e-002 >> 0 KSP Residual norm 9.965769844003e-002 >> 0 KSP preconditioned resid norm 9.965769844003e-002 true resid norm >> 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.919247724727e-009 >> 1 KSP preconditioned resid norm 4.919247724727e-009 true resid norm >> 2.073848008334e-002 ||Ae||/||Ax|| 2.932863979685e-001 >> 1 SNES Function norm 6.402041310994e-002 >> 0 KSP Residual norm 8.969192038029e-002 >> 0 KSP preconditioned resid norm 8.969192038029e-002 true resid norm >> 6.402041310994e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.072962844649e-009 >> 1 KSP preconditioned resid norm 4.072962844649e-009 true resid norm >> 2.116458841832e-002 ||Ae||/||Ax|| 3.305912503559e-001 >> 2 SNES Function norm 5.850358277995e-002 >> 0 KSP Residual norm 8.072272129834e-002 >> 0 KSP preconditioned resid norm 8.072272129834e-002 true resid norm >> 5.850358277995e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.535693836801e-009 >> 1 KSP preconditioned resid norm 1.535693836801e-009 true resid norm >> 1.378413836547e-002 ||Ae||/||Ax|| 2.356118670085e-001 >> 3 SNES Function norm 5.377276379928e-002 >> 0 KSP Residual norm 7.265044334812e-002 >> 0 KSP preconditioned resid norm 7.265044334812e-002 true resid norm >> 5.377276379928e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.196914267894e-009 >> 1 KSP preconditioned resid norm 8.196914267894e-009 true resid norm >> 1.643681592299e-002 ||Ae||/||Ax|| 3.056717706448e-001 >> 4 SNES Function norm 4.957825627064e-002 >> 0 KSP Residual norm 6.538539498475e-002 >> 0 KSP preconditioned resid norm 6.538539498475e-002 true resid norm >> 4.957825627064e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.605278143372e-009 >> 1 KSP preconditioned resid norm 1.605278143372e-009 true resid norm >> 1.365749144340e-002 ||Ae||/||Ax|| 2.754734125550e-001 >> 5 SNES Function norm 4.576573391352e-002 >> 0 KSP Residual norm 5.884685187863e-002 >> 0 KSP preconditioned resid norm 5.884685187863e-002 true resid norm >> 4.576573391352e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.917790130377e-009 >> 1 KSP preconditioned resid norm 3.917790130377e-009 true resid norm >> 2.918137698804e-002 ||Ae||/||Ax|| 6.376250196967e-001 >> 6 SNES Function norm 4.224348383506e-002 >> 0 KSP Residual norm 5.296216341013e-002 >> 0 KSP preconditioned resid norm 5.296216341013e-002 true resid norm >> 4.224348383506e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.384279752582e-009 >> 1 KSP preconditioned resid norm 1.384279752582e-009 true resid norm >> 8.645143617596e-003 ||Ae||/||Ax|| 2.046503468168e-001 >> 7 SNES Function norm 3.895875122953e-002 >> 0 KSP Residual norm 4.766594475257e-002 >> 0 KSP preconditioned resid norm 4.766594475257e-002 true resid norm >> 3.895875122953e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.442999105246e-009 >> 1 KSP preconditioned resid norm 5.442999105246e-009 true resid norm >> 3.199366262059e-002 ||Ae||/||Ax|| 8.212188946227e-001 >> 8 SNES Function norm 3.588203219083e-002 >> 0 KSP Residual norm 4.289934859308e-002 >> 0 KSP preconditioned resid norm 4.289934859308e-002 true resid norm >> 3.588203219083e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.069272581944e-009 >> 1 KSP preconditioned resid norm 5.069272581944e-009 true resid norm >> 3.315026802067e-002 ||Ae||/||Ax|| 9.238681868510e-001 >> 9 SNES Function norm 3.299674865285e-002 >> 0 KSP Residual norm 3.860941169595e-002 >> 0 KSP preconditioned resid norm 3.860941169595e-002 true resid norm >> 3.299674865285e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.314510318621e-009 >> 1 KSP preconditioned resid norm 2.314510318621e-009 true resid norm >> 1.422355468145e-002 ||Ae||/||Ax|| 4.310592789335e-001 >> 10 SNES Function norm 3.020092421335e-002 >> 0 KSP Residual norm 3.457428537321e-002 >> 0 KSP preconditioned resid norm 3.457428537321e-002 true resid norm >> 3.020092421335e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.171225245965e-009 >> 1 KSP preconditioned resid norm 3.171225245965e-009 true resid norm >> 6.700877909403e-003 ||Ae||/||Ax|| 2.218765843742e-001 >> 11 SNES Function norm 2.761432326644e-002 >> 0 KSP Residual norm 3.100402216444e-002 >> 0 KSP preconditioned resid norm 3.100402216444e-002 true resid norm >> 2.761432326644e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.317753799744e-009 >> 1 KSP preconditioned resid norm 2.317753799744e-009 true resid norm >> 1.143024510909e-002 ||Ae||/||Ax|| 4.139245057287e-001 >> 12 SNES Function norm 2.483228477543e-002 >> 0 KSP Residual norm 2.716039584199e-002 >> 0 KSP preconditioned resid norm 2.716039584199e-002 true resid norm >> 2.483228477543e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.024860175617e-009 >> 1 KSP preconditioned resid norm 1.024860175617e-009 true resid norm >> 5.071961436716e-003 ||Ae||/||Ax|| 2.042486820115e-001 >> 13 SNES Function norm 2.155599009420e-002 >> 0 KSP Residual norm 2.247739022940e-002 >> 0 KSP preconditioned resid norm 2.247739022940e-002 true resid norm >> 2.155599009420e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.231896568498e-009 >> 1 KSP preconditioned resid norm 1.231896568498e-009 true resid norm >> 2.253717888475e-003 ||Ae||/||Ax|| 1.045518150002e-001 >> 14 SNES Function norm 1.758159262430e-002 >> 0 KSP Residual norm 1.640654408633e-002 >> 0 KSP preconditioned resid norm 1.640654408633e-002 true resid norm >> 1.758159262430e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.203409491551e-009 >> 1 KSP preconditioned resid norm 1.203409491551e-009 true resid norm >> 3.340286893989e-003 ||Ae||/||Ax|| 1.899877312236e-001 >> 15 SNES Function norm 1.275874713861e-002 >> 0 KSP Residual norm 8.737890404296e-003 >> 0 KSP preconditioned resid norm 8.737890404296e-003 true resid norm >> 1.275874713861e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.862616851249e-010 >> 1 KSP preconditioned resid norm 3.862616851249e-010 true resid norm >> 9.158071400861e-004 ||Ae||/||Ax|| 7.177876715771e-002 >> 16 SNES Function norm 5.367994969686e-003 >> 0 KSP Residual norm 1.873266786098e-009 >> 0 KSP preconditioned resid norm 1.873266786098e-009 true resid norm >> 5.367994969686e-003 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.236612780542e-017 >> 1 KSP preconditioned resid norm 9.236612780542e-017 true resid norm >> 4.487656305545e-010 ||Ae||/||Ax|| 8.360023306445e-008 >> 17 SNES Function norm 1.843858578272e-008 >> 0 KSP Residual norm 3.418191528615e-015 >> 0 KSP preconditioned resid norm 3.418191528615e-015 true resid norm >> 1.843858578272e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.091028447697e-022 >> 1 KSP preconditioned resid norm 2.091028447697e-022 true resid norm >> 9.356082852780e-016 ||Ae||/||Ax|| 5.074186796662e-008 >> 18 SNES Function norm 1.582175998776e-008 >> STEP 1 (Newton iterations: 18) >> >> 0 SNES Function norm 7.071067811866e-002 >> 0 KSP Residual norm 9.965760716669e-002 >> 0 KSP preconditioned resid norm 9.965760716669e-002 true resid norm >> 7.071067811866e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.158690871520e-009 >> 1 KSP preconditioned resid norm 4.158690871520e-009 true resid norm >> 1.356168089543e-002 ||Ae||/||Ax|| 1.917911305090e-001 >> 1 SNES Function norm 6.402155946870e-002 >> 0 KSP Residual norm 8.969183775926e-002 >> 0 KSP preconditioned resid norm 8.969183775926e-002 true resid norm >> 6.402155946870e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.922035131300e-009 >> 1 KSP preconditioned resid norm 7.922035131300e-009 true resid norm >> 4.173737132742e-002 ||Ae||/||Ax|| 6.519268145573e-001 >> 2 SNES Function norm 5.850726192206e-002 >> 0 KSP Residual norm 8.072264774465e-002 >> 0 KSP preconditioned resid norm 8.072264774465e-002 true resid norm >> 5.850726192206e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.356474211542e-009 >> 1 KSP preconditioned resid norm 4.356474211542e-009 true resid norm >> 2.083500604436e-002 ||Ae||/||Ax|| 3.561097436437e-001 >> 3 SNES Function norm 5.377933295280e-002 >> 0 KSP Residual norm 7.265037686612e-002 >> 0 KSP preconditioned resid norm 7.265037686612e-002 true resid norm >> 5.377933295280e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.772318530630e-009 >> 1 KSP preconditioned resid norm 8.772318530630e-009 true resid norm >> 5.284958913103e-002 ||Ae||/||Ax|| 9.827118751623e-001 >> 4 SNES Function norm 4.958751191600e-002 >> 0 KSP Residual norm 6.538533386375e-002 >> 0 KSP preconditioned resid norm 6.538533386375e-002 true resid norm >> 4.958751191600e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.128741903850e-009 >> 1 KSP preconditioned resid norm 1.128741903850e-009 true resid norm >> 2.526410501148e-003 ||Ae||/||Ax|| 5.094852319729e-002 >> 5 SNES Function norm 4.577729047509e-002 >> 0 KSP Residual norm 5.884679667348e-002 >> 0 KSP preconditioned resid norm 5.884679667348e-002 true resid norm >> 4.577729047509e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.763627011382e-009 >> 1 KSP preconditioned resid norm 5.763627011382e-009 true resid norm >> 1.051243252216e-002 ||Ae||/||Ax|| 2.296429608013e-001 >> 6 SNES Function norm 4.225677503639e-002 >> 0 KSP Residual norm 5.296211439112e-002 >> 0 KSP preconditioned resid norm 5.296211439112e-002 true resid norm >> 4.225677503639e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.022912340341e-009 >> 1 KSP preconditioned resid norm 1.022912340341e-009 true resid norm >> 3.573383237647e-003 ||Ae||/||Ax|| 8.456355778617e-002 >> 7 SNES Function norm 3.897321226706e-002 >> 0 KSP Residual norm 4.766590050155e-002 >> 0 KSP preconditioned resid norm 4.766590050155e-002 true resid norm >> 3.897321226706e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.528025220977e-009 >> 1 KSP preconditioned resid norm 3.528025220977e-009 true resid norm >> 1.960313404416e-002 ||Ae||/||Ax|| 5.029899488355e-001 >> 8 SNES Function norm 3.589720610234e-002 >> 0 KSP Residual norm 4.289930864255e-002 >> 0 KSP preconditioned resid norm 4.289930864255e-002 true resid norm >> 3.589720610234e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.133432759880e-009 >> 1 KSP preconditioned resid norm 4.133432759880e-009 true resid norm >> 1.940005023620e-002 ||Ae||/||Ax|| 5.404334304149e-001 >> 9 SNES Function norm 3.301230929196e-002 >> 0 KSP Residual norm 3.860937624684e-002 >> 0 KSP preconditioned resid norm 3.860937624684e-002 true resid norm >> 3.301230929196e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.211304264406e-010 >> 1 KSP preconditioned resid norm 8.211304264406e-010 true resid norm >> 7.328958637952e-003 ||Ae||/||Ax|| 2.220068451781e-001 >> 10 SNES Function norm 3.030900587812e-002 >> 0 KSP Residual norm 3.474843739750e-002 >> 0 KSP preconditioned resid norm 3.474843739750e-002 true resid norm >> 3.030900587812e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.265840938498e-009 >> 1 KSP preconditioned resid norm 3.265840938498e-009 true resid norm >> 1.358870142093e-002 ||Ae||/||Ax|| 4.483387371916e-001 >> 11 SNES Function norm 2.718409957830e-002 >> 0 KSP Residual norm 3.013614572851e-002 >> 0 KSP preconditioned resid norm 3.013614572851e-002 true resid norm >> 2.718409957830e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.652744245097e-009 >> 1 KSP preconditioned resid norm 1.652744245097e-009 true resid norm >> 1.202105117910e-002 ||Ae||/||Ax|| 4.422089149754e-001 >> 12 SNES Function norm 2.437348813670e-002 >> 0 KSP Residual norm 2.629377332723e-002 >> 0 KSP preconditioned resid norm 2.629377332723e-002 true resid norm >> 2.437348813670e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.959255153953e-009 >> 1 KSP preconditioned resid norm 3.959255153953e-009 true resid norm >> 2.578989386428e-002 ||Ae||/||Ax|| 1.058112557367e+000 >> 13 SNES Function norm 2.199784341474e-002 >> 0 KSP Residual norm 2.328182605620e-002 >> 0 KSP preconditioned resid norm 2.328182605620e-002 true resid norm >> 2.199784341474e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.915626892590e-010 >> 1 KSP preconditioned resid norm 8.915626892590e-010 true resid norm >> 6.263373898192e-003 ||Ae||/||Ax|| 2.847267243477e-001 >> 14 SNES Function norm 1.833996742657e-002 >> 0 KSP Residual norm 1.794186796967e-002 >> 0 KSP preconditioned resid norm 1.794186796967e-002 true resid norm >> 1.833996742657e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.694980794482e-009 >> 1 KSP preconditioned resid norm 1.694980794482e-009 true resid norm >> 2.372076451714e-003 ||Ae||/||Ax|| 1.293391856453e-001 >> 15 SNES Function norm 1.384131369872e-002 >> 0 KSP Residual norm 1.074320273869e-002 >> 0 KSP preconditioned resid norm 1.074320273869e-002 true resid norm >> 1.384131369872e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 7.118408455169e-010 >> 1 KSP preconditioned resid norm 7.118408455169e-010 true resid norm >> 4.592913592019e-003 ||Ae||/||Ax|| 3.318264213927e-001 >> 16 SNES Function norm 8.126551489387e-003 >> 0 KSP Residual norm 2.817712852237e-009 >> 0 KSP preconditioned resid norm 2.817712852237e-009 true resid norm >> 8.126551489387e-003 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.239560797627e-016 >> 1 KSP preconditioned resid norm 2.239560797627e-016 true resid norm >> 1.053395037920e-009 ||Ae||/||Ax|| 1.296238680448e-007 >> 17 SNES Function norm 4.871218193429e-008 >> 0 KSP Residual norm 1.123675925695e-014 >> 0 KSP preconditioned resid norm 1.123675925695e-014 true resid norm >> 4.871218193429e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.307928330547e-022 >> 1 KSP preconditioned resid norm 5.307928330547e-022 true resid norm >> 2.893354456541e-015 ||Ae||/||Ax|| 5.939693813026e-008 >> 18 SNES Function norm 5.035279029783e-008 >> STEP 2 (Newton iterations: 18) >> >> diverged reason: -6 >> >> >> Rong >> >>> ----- Original Message ----- From: "Barry Smith" >>> To: "PETSc users list" >>> Sent: Wednesday, May 11, 2011 11:05 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>> >>> Ok, the linear solver looks like it is working ok. The likely problem is >>> that Jacobian does not match the function evaluation. >>> >>> Run the same thing but with the additional option -snes_mf_operator >>> >>> Then run with -snes_type test (instead of -snes_mf_operator). >>> >>> Barry >>> >>> On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: >>> >>>> Dear Barry, here is the output using -pc_type >>>> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>>> the attached is the same and for clear reference. Thanks again for >>>> helps. >>>> >>>> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >>>> 0 SNES Function norm 7.071067811865e-002 >>>> 0 KSP Residual norm 9.965778978387e-002 >>>> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >>>> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.823187455811e-017 >>>> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >>>> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >>>> 1 SNES Function norm 6.401926523423e-002 >>>> 0 KSP Residual norm 8.969200212486e-002 >>>> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >>>> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.106757475780e-016 >>>> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >>>> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >>>> 2 SNES Function norm 5.849992149767e-002 >>>> 0 KSP Residual norm 8.072279488157e-002 >>>> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >>>> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.268750073799e-017 >>>> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >>>> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >>>> 3 SNES Function norm 5.376618503592e-002 >>>> 0 KSP Residual norm 7.265050969883e-002 >>>> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >>>> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.677655733356e-017 >>>> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >>>> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >>>> 4 SNES Function norm 4.956894354459e-002 >>>> 0 KSP Residual norm 6.538545411661e-002 >>>> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >>>> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 9.557004153175e-017 >>>> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >>>> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >>>> 5 SNES Function norm 4.575418613137e-002 >>>> 0 KSP Residual norm 5.884690496914e-002 >>>> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >>>> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 5.470969262115e-017 >>>> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >>>> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >>>> 6 SNES Function norm 4.223022245585e-002 >>>> 0 KSP Residual norm 5.296221144636e-002 >>>> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >>>> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 8.255198782390e-017 >>>> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >>>> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >>>> 7 SNES Function norm 3.894430065910e-002 >>>> 0 KSP Residual norm 4.766598785088e-002 >>>> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >>>> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 3.322615478395e-017 >>>> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >>>> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >>>> 8 SNES Function norm 3.586683371135e-002 >>>> 0 KSP Residual norm 4.289938708067e-002 >>>> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >>>> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.181358328498e-017 >>>> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >>>> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >>>> 9 SNES Function norm 3.298130202025e-002 >>>> 0 KSP Residual norm 3.860944676473e-002 >>>> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >>>> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 4.635174776374e-017 >>>> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >>>> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >>>> 10 SNES Function norm 3.027806208930e-002 >>>> 0 KSP Residual norm 3.474850078591e-002 >>>> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >>>> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.389914053685e-017 >>>> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >>>> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >>>> 11 SNES Function norm 2.749422924729e-002 >>>> 0 KSP Residual norm 3.081350823297e-002 >>>> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >>>> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.640567497647e-017 >>>> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >>>> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >>>> 12 SNES Function norm 2.437488247885e-002 >>>> 0 KSP Residual norm 2.633007441879e-002 >>>> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >>>> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.772331460094e-017 >>>> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >>>> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >>>> 13 SNES Function norm 2.079664278637e-002 >>>> 0 KSP Residual norm 2.104738289397e-002 >>>> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >>>> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.650632708670e-017 >>>> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >>>> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >>>> 14 SNES Function norm 1.657344626858e-002 >>>> 0 KSP Residual norm 1.454141853505e-002 >>>> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >>>> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 1.129401160070e-017 >>>> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >>>> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >>>> 15 SNES Function norm 1.484243752612e-002 >>>> 0 KSP Residual norm 5.241948491751e-009 >>>> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >>>> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 2.729506849025e-024 >>>> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >>>> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >>>> 16 SNES Function norm 2.828002157497e-008 >>>> 0 KSP Residual norm 6.042518362322e-015 >>>> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >>>> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >>>> 1 KSP Residual norm 6.272441346127e-030 >>>> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >>>> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >>>> 17 SNES Function norm 2.960967020289e-008 >>>> STEP 0 (Newton iterations: 17) >>>> >>>> diverged reason: -6 >>>> >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Wednesday, May 11, 2011 2:54 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >>>> and >>>> send the outputs >>>> >>>> >>>> Barry >>>> >>>> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >>>> >>>>> by the way, the increment size is like that >>>>> for a 100 lengh model, the increment is set to 0.05, >>>>> the engineering strain is around 5%% per load step. >>>>> This is already too small increment size for a large deformation >>>>> analysis. >>>>> a 0.5 increment size leads to both linear search and trust region >>>>> failed. >>>>> linear search failed for 0.05 while trust region converges with 17 >>>>> Newton >>>>> iterations each load step. >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Tian(ICT)" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 11:37 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>>> First, thanks again, the issue was gone. >>>>>> >>>>>> I just followed up with some test results. >>>>>> I have tested SNES using one finite element for a geometric large >>>>>> deformation problem. >>>>>> Those are just the very early test results so they may be not telling >>>>>> what happened exactly. >>>>>> For the displacement controlled load, I found that convergence is >>>>>> much >>>>>> slower than that of force loading. >>>>>> Even worse, linear search is so sensitive to the displacement >>>>>> increment >>>>>> and diverged no matter what the increment size was used (too small >>>>>> incremnt also led to diverged soloution (-6 reason), trust region >>>>>> works >>>>>> well in the sense of not sensitive to the displacement increment, but >>>>>> during each load step, it requires around ten to several tens of >>>>>> Newton >>>>>> interations whereas for the force loading case and the almost same >>>>>> amount of deformation, this is normally 3. This is against my >>>>>> expectation. Any hint? >>>>>> >>>>>> Rong >>>>>> >>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>> To: "PETSc users list" >>>>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>> >>>>>> >>>>>> >>>>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>>>> >>>>>>> Dear Barry, Thanks a lot for quick answering. >>>>>>> I checked the development documents and found the new version of >>>>>>> MatZeroRows() does support the nonzero prescribed boundary >>>>>>> conditions. >>>>>>> >>>>>>> I followed up with more details. >>>>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using >>>>>>> SNES >>>>>>> solvers. >>>>>>> I used a displacement-controlled load (as this type of loading works >>>>>>> well for all cases). >>>>>>> This is the reason the nonzero prescribed boundary came up. >>>>>>> >>>>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>>>> nonzero prescribed boundary. >>>>>>> In FormFunction, I modified the solution to the known solution(this >>>>>>> should not be necessary as the modified Jacobian and rhs should give >>>>>>> the prescribed solution also) >>>>>> >>>>>> You should not do it this way. See below. >>>>>>> >>>>>>> Now I found another issue, no matter if I prescried the solution or >>>>>>> not >>>>>>> in FormFunction, >>>>>>> SNES solver always call FormFunction and never call FormJacobian. >>>>>> >>>>>> The only reason it would not call FormJacobian is if decided that the >>>>>> residual norm was small enough before any Newton steps; for example >>>>>> if >>>>>> the FormFunction() computed exactly the zero function initially. When >>>>>> you run with -snes_monitor -ksp_monitor what does it print for >>>>>> residual >>>>>> norms. >>>>>> >>>>>>> Of course the solver finally diverged or converged to a zero >>>>>>> solution. >>>>>>> >>>>>>> So my quick follow up question is How a displacement-controled load >>>>>>> is >>>>>>> done corrently in Petsc 2.3.3? >>>>>> >>>>>> To do it in 2.3.3 simply have for those components of F() the formula >>>>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() >>>>>> for >>>>>> those rows >>>>>> >>>>>> We strongly urge you to upgrade to the latest PETSc before doing >>>>>> anything further. >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>>> >>>>>>> Rong >>>>>>> >>>>>>> ----- Original Message ----- From: "Barry Smith" >>>>>>> >>>>>>> To: "PETSc users list" >>>>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>>>> >>>>>>> >>>>>>> >>>>>>> In petsc-dev >>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>>>> modified the calling sequence for MatZeroRows() so that it can >>>>>>> automatically adjust the appropriate right hand side values for the >>>>>>> zeroed rows to support zero or non-zero prescribed boundary >>>>>>> conditions >>>>>>> easily. >>>>>>> >>>>>>> Barry >>>>>>> >>>>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>>>> >>>>>>>> Dear all, >>>>>>>> >>>>>>>> I got this question long ago and searched the prior posting but did >>>>>>>> not find the solution. >>>>>>>> The question is about nonzero prescribed boundary condition. >>>>>>>> My understanding is that MatZeroRows() works only for zero >>>>>>>> prescribed >>>>>>>> value, not non-zero value. >>>>>>>> For the non-zero values, we have to remove the rows associated with >>>>>>>> the boundary, but this >>>>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>>>> also be removed. >>>>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>>>> prescribed boundary and if so how to do it simply? >>>>>>>> >>>>>>>> Rong >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >> > > From sylbar.vainbot at gmail.com Thu May 12 01:01:40 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Wed, 11 May 2011 23:01:40 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: Hi Barry, I have tested the direct solver with the shell matrix and it works very well. In my code, the matrix-multiply operation is implemented with the function "matrixantiplanefv". I am now trying to do it with multi-grid. I am keeping a direct solver solely for the coarse-level. I will do this coarse solve matrix-free with the function "matrixantiplanecoarsefv". I am also setting up all the other operations (residual, smoother, restriction, interpolation) with matrix-free methods. The setup is done in the subroutine "green_init_mg". First I set up a PCMG preconditionner: CALL KSPCreate(PETSC_COMM_WORLD,c%ksp,ierr); CALL KSPGetPC(c%ksp,c%pc,ierr); CALL PCSetType(c%pc,PCMG,ierr); CALL PCMGSetLevels(c%pc,c%nlevels,PETSC_NULL_OBJECT,ierr); CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); CALL PCMGSetCycleType(c%pc,PC_MG_CYCLE_V,ierr); CALL PCMGSetNumberSmoothUp(c%pc,1,ierr); CALL PCMGSetNumberSmoothDown(c%pc,1,ierr); Then I set up the coarse solver: CALL MatCreateShell(PETSC_COMM_WORLD, & cinfo%zm*cinfo%ym*cinfo%xm*info%dof,cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof, & cinfo%mz*cinfo%my*cinfo%mx*info%dof,cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof, & PETSC_NULL,c%cA,ierr); CALL MatShellSetOperation(c%cA,MATOP_MULT,matrixantiplanecoarsefv,ierr); CALL MatAssemblyBegin(c%cA,MAT_FINAL_ASSEMBLY,ierr); CALL MatAssemblyEnd(c%cA,MAT_FINAL_ASSEMBLY,ierr); CALL KSPSetOperators(c%cksp,c%cA,c%cA,DIFFERENT_NONZERO_PATTERN,ierr); CALL KSPSetUp(c%cksp,ierr); CALL PCMGGetCoarseSolve(c%pc,c%cksp,ierr); Then I set up the interpolation and restriction: DO l=1,c%nlevels-1 ... ! shell matrix MxN with M>N CALL MatCreateShell(PETSC_COMM_WORLD, & cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof,minfo%zm*minfo%ym*minfo%xm*minfo%dof, & cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof,minfo%mz*minfo%my*minfo%mx*minfo%dof, & PETSC_NULL,c%interp(l),ierr); CALL MatShellSetOperation(c%interp(l),MATOP_MULT,matrixinterp,ierr); CALL MatAssemblyBegin(c%interp(l),MAT_FINAL_ASSEMBLY,ierr); CALL MatAssemblyEnd(c%interp(l),MAT_FINAL_ASSEMBLY,ierr); ! shell matrix MxN with M: > > > ? Barry > > > On May 10, 2011, at 9:20 PM, Sylvain Barbot wrote: > >> Dear Jed, >> >> Thank you for your previous comments. I am still trying to design a >> multigrid preconditionner for the Navier's equation of elasticity. I >> have defined shell matrices for the interpolation, restriction, >> smoothing, residual calculation and the fine-level solver (see >> attached my_green.f90 file with stub functions). I thought that all >> these shell matrices would be enough to do the job, but upon running >> KSPSetup(), I get the following error: >> >> [0]PETSC ERROR: No support for this operation for this object type! >> [0]PETSC ERROR: This matrix type, shell, does not support getting >> diagonal block! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 >> 14:42:02 CDT 2010 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c >> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c >> >> regarding your previous comments: "For Jacobi, you can have your >> MatShell implement MatGetDiagonal and it will work. (...) If you want >> to use a multiplicative relaxation like SOR, you would have to >> implement it yourself.", I actually want to use SOR with my smoother >> shell matrix. What am I doing wrong to make petsc still believe I want >> to use Jacobi? >> > > ?By default, it is using block Jacobi on each level (except the coarsest). You need to tell it to use the relaxation you have coded. For example if your shell matrix provides a SOR/relaxation method then you need to use -mg_levels_pc_type sor ?and it will call that on each level instead of block Jacobi. In looking at your code I see the only operation your provide your shell matrices are matrix-vector multiply. Therefore you cannot use SOR/relaxation. The only think you can use is GMRES (or some other Krylov method) with no preconditioner as the smoother. For this you can use -mg_levels_pc_type none -mg_levels_ksp_type gmres > > ? In your message ?you say ?"I actually want to use SOR with my smoother" to do this you need to provide the smoother function and set it with MatShellSetOperations(). > > >> another related question: with a direct solver, one needs to assign a >> shell matrix with a KSP object (KSPSetOperators); is it still required >> when using multigrid? if I don't do so, I get the following error: >> > ?Where are you using the direct solver for, on the entire linear system or on the coarse mesh? In either case you NEED to provide an explicit (not shell) sparse matrix otherwise a direct solver cannot be used. From the message below it looks like you did not provide a matrix somewhere. If you don't provide a matrix a direct solver cannot be used. > > > ? Barry > >> [0]PETSC ERROR: Null argument, when expecting valid pointer! >> [0]PETSC ERROR: Null Object: Parameter # 1! >> [0]PETSC ERROR: MatGetVecs() line 7265 in src/mat/interface/matrix.c >> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >> [0]PETSC ERROR: KSPSetUp() line 199 in src/ksp/ksp/interface/itfunc.c >> >> however, I don't understand why I would need this matrix in a >> multigrid framework. >> >> Is there any example out there of using PCMG with shell matrices? >> >> Thanks in advance for your response. >> Best wishes, >> Sylvain Barbot >> >> >> 2010/10/7 Jed Brown : >>> On Thu, Oct 7, 2010 at 11:45, Sylvain Barbot >>> wrote: >>>> >>>> questions: >>>> 1) is it at all possible to specify this mode of operation, from the >>>> finest to the coarser level, and back? Any examples out there? >>> >>> -pc_mg_type multiplicative, or >>> ? PCMGSetType(pc,PC_MG_MULTIPLICATIVE); >>> >>>> >>>> 2) is it readily possible to use matrix shells with DMMG? I imagine >>>> the Jacobian matrix may simply be provided as a matrix shell. Is there >>>> any examples of multi-grid methods with shell matrices online? >>> >>> You can do this, but you will have to define a custom smoother. ?For Jacobi, >>> you can have your MatShell implement MatGetDiagonal and it will work. ?I >>> thought you could implement MatGetDiagonalBlock for PBJacobi, but it's not >>> currently written that way (though it should be and that would be an easy >>> change to make). ?If you want to use a multiplicative relaxation like SOR, >>> you would have to implement it yourself. ?If you need something like ILU for >>> a smoother, then you will have to pay for a matrix. ?Note that one >>> possibility is to assemble a matrix on all but the finest level so you can >>> use stronger smoothers there, then make do with Jacobi on the finest level. >>> >>>> >>>> 3) to deal with non-uniform sampling: can I provide the coordinates of >>>> the finest grid with DASetCoordinates, then expect DMMG to provide the >>>> subsampled coordinates at the coarser levels? >>> >>> Currently no, you have to set them on each level. ?Perhaps you could do this >>> by rolling a loop over levels and applying MatRestrict using the restriction >>> matrix from DMMG (this might not be the sampling that you want). >>> Jed >> > > From sylbar.vainbot at gmail.com Thu May 12 01:12:34 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Wed, 11 May 2011 23:12:34 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: Dear Jed, During my recent visit to ETH, I talked at length about multi-grid with Dave May who warned me about the issues of large coefficient-contrasts. Most of my problems of interest for tectonophysics and earthquake simulations are cases of relatively smooth variations in elastic moduli. So I am not too worried about this aspect of the problem. I appreciate your advice about trying simpler solutions first. I have tested at length direct solvers of 2-D and 3-D problems of elastic deformation and I am quite happy with the results. My primary concern now is computation speed, especially for 3-D problems, where i have of the order 512^3 degrees of freedom. I was planning to test Jacobi and SOR smoothers. Is there another smoother you recommend for this kind of problem? Thanks, Sylvain 2011/5/11 Jed Brown : > On Wed, May 11, 2011 at 04:20, Sylvain Barbot > wrote: >> >> I am still trying to design a >> multigrid preconditionner for the Navier's equation of elasticity. > > I have heard, through an external source, that you have large jumps in both > Young's modulus and Poisson ratio that are not grid aligned, including > perhaps thin structures that span a large part of the domain. Such problems > are pretty hard, so I suggest you focus on robustness and do not worry about > low-memory implementation at this point. That is, you should assemble the > matrices in a usual PETSc format instead of using MatShell to do everything > matrix-free. This gives you access to much stronger smoothers. > After you find a scheme that is robust enough for your purposes, _then_ you > can make it low-memory by replacing some assembled matrices by MatShell. To > realize most of the possible memory savings, it should be sufficient to do > this on the finest level only. From jed at 59A2.org Thu May 12 05:37:07 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 12 May 2011 12:37:07 +0200 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: On Thu, May 12, 2011 at 08:12, Sylvain Barbot wrote: > My primary concern now is computation speed, especially for > 3-D problems, where i have of the order 512^3 degrees of freedom. I > was planning to test Jacobi and SOR smoothers. Is there another > smoother you recommend for this kind of problem? > It's worth trying ILU as a smoother, especially if you are using a stretched mesh. What discretization are you using? I still recommend just writing the matrix assembly and experimenting (you can do everything on the command line). Then, if the problem of interest responds well to SOR, you can easily put in an option to use matrix-free. That is, you would have one code with a runtime option of which levels to assemble versus handle matrix-free. The fastest choice will almost certainly depend on the problem and on the hardware that you are running it on, so it's well worth having both options available. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 12 05:41:41 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 12 May 2011 12:41:41 +0200 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: <71C9B313096748028405BE22EBAF85CD@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <1302D684D3644298A3F2528BDEE08857@rti458laptop> <32DB8456522B4FABB725933C113D66D1@rti458laptop> <03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov> <71C9B313096748028405BE22EBAF85CD@rti458laptop> Message-ID: On Thu, May 12, 2011 at 06:35, Tian(ICT) wrote: > Without any option, SNES converges never calling FormJacobian, > and the solution seems correct. > > What is the reason for that the SNES solver never calls FormJacobian (mf > option was not used)? > Please send the following: 1. The code where you configure the solver, everything from SNESCreate to SNESSolve 2. The complete command line that you run with. 3. Output when you add "-snes_view -options_table" to the command line -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Thu May 12 06:01:24 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Thu, 12 May 2011 19:01:24 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop><1302D684D3644298A3F2528BDEE08857@rti458laptop><32DB8456522B4FABB725933C113D66D1@rti458laptop><03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov><71C9B313096748028405BE22EBAF85CD@rti458laptop> Message-ID: char msg[125]; SNES snes; ApplicationCtx ctx; PetscErrorCode ierr; PetscReal abstol,rtol,stol; PetscInt its,maxit,maxf; PetscReal none = -1.0; PetscScalar zero = 0; ctx.pDomain = this; pPhyDomain->Initialize(ASSEMBLY_FREE_MESH); PetscInt N = pPhyDomain->GetEquCount(); Mat J; Vec u,r; SNESCreate(PETSC_COMM_WORLD,&snes); VecCreate(PETSC_COMM_WORLD,&u); VecSetSizes(u,N,N); VecSetFromOptions(u); VecDuplicate(u,&r); VecDuplicate(u,&ctx.oldu); SNESSetFunction(snes,r,FormFunction,&ctx); MatCreate(PETSC_COMM_WORLD,&J); MatSetSizes(J,PETSC_DECIDE,PETSC_DECIDE,N,N); MatSetFromOptions(J); SNESSetJacobian(snes,J,J,FormJacobian,&ctx); PetscObjectSetName((PetscObject)u,"Approximate Solution"); SNESSetFromOptions(snes); ierr = SNESGetTolerances(snes,&abstol,&rtol,&stol,&maxit,&maxf);CHKERRQ(ierr); ierr = PetscPrintf(PETSC_COMM_WORLD,"atol=%G, rtol=%G, stol=%G, maxit=%D, maxf=%D\n",abstol,rtol,stol,maxit,maxf);CHKERRQ(ierr); //incremental lambda int step = 0; ctx.dlambda = 1.0/ctx.pDomain->contrl.loadSteps; for(double lambda = ctx.dlambda; lambda <= 1.0; lambda += ctx.dlambda){ ctx.lambda = lambda * ctx.pDomain->contrl.loadScaler; //initialize previous displacment VecSet(ctx.oldu,0.0); ierr = FormInitialGuess(u);CHKERRQ(ierr); //solve ierr = SNESSolve(snes,PETSC_NULL,u);CHKERRQ(ierr); SNESGetIterationNumber(snes,&its); sprintf(msg,"STEP %d (Newton iterations: %d)\n\n",step, its); Log::Send(Log::Info, msg); ierr = PetscPrintf(PETSC_COMM_WORLD,"STEP %D (Newton iterations: %D)\n\n",step, its);CHKERRQ(ierr); //postprocess if(step % ctx.pDomain->contrl.outputFrequency == 0){ pPhyDomain->WriteConfiguration(step); pPhyDomain->WriteResultFiles(step); } //convergence test SNESConvergedReason reason; SNESGetConvergedReason(snes, &reason); if(reason < 0){//diverged Log::Send(Log::Info, " calculation diverged"); cout << "diverged reason: " << reason << endl; break; } //advance step ++; } VecDestroy(r); MatDestroy(J); VecDestroy(u); SNESDestroy(snes); PetscFunctionReturn(0); } total number of function evaluations=4 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonaliza tion with no iterative refinement GMRES: happy breakdown tolerance 1e-030 maximum iterations=10000, initial guess is zero tolerances: relative=1e-005, absolute=1e-050, divergence=10000 left preconditioning PC Object: type: ilu ILU: 0 levels of fill ILU: factor fill ratio allocated 1 ILU: tolerance for zero pivot 1e-012 out-of-place factorization matrix ordering: natural ILU: factor fill ratio needed 1 Factored matrix follows Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=840 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=3480 using I-node routines: found 15 nodes, limit used is 5 STEP 3 (Newton iterations: 3) SNES Object: type: ls line search variant: SNESLineSearchCubic alpha=0.0001, maxstep=1e+008, steptol=1e-012 maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-008, absolute=1e-050, solution=1e-008 total number of linear solver iterations=3 total number of function evaluations=4 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonaliza tion with no iterative refinement GMRES: happy breakdown tolerance 1e-030 maximum iterations=10000, initial guess is zero The output: C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_view -options_tableclear atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 SNES Object: type: ls line search variant: SNESLineSearchCubic alpha=0.0001, maxstep=1e+008, steptol=1e-012 maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-008, absolute=1e-050, solution=1e-008 total number of linear solver iterations=3 total number of function evaluations=4 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonaliza tion with no iterative refinement GMRES: happy breakdown tolerance 1e-030 maximum iterations=10000, initial guess is zero tolerances: relative=1e-005, absolute=1e-050, divergence=10000 left preconditioning PC Object: type: ilu ILU: 0 levels of fill ILU: factor fill ratio allocated 1 ILU: tolerance for zero pivot 1e-012 out-of-place factorization matrix ordering: natural ILU: factor fill ratio needed 1 Factored matrix follows Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=840 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=1320 using I-node routines: found 15 nodes, limit used is 5 STEP 0 (Newton iterations: 3) SNES Object: type: ls line search variant: SNESLineSearchCubic alpha=0.0001, maxstep=1e+008, steptol=1e-012 maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-008, absolute=1e-050, solution=1e-008 total number of linear solver iterations=3 total number of function evaluations=4 KSP Object: type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonaliza tion with no iterative refinement GMRES: happy breakdown tolerance 1e-030 maximum iterations=10000, initial guess is zero tolerances: relative=1e-005, absolute=1e-050, divergence=10000 left preconditioning PC Object: type: ilu ILU: 0 levels of fill ILU: factor fill ratio allocated 1 ILU: tolerance for zero pivot 1e-012 out-of-place factorization matrix ordering: natural ILU: factor fill ratio needed 1 Factored matrix follows Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=840 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=24, cols=24 total: nonzeros=392, allocated nonzeros=2040 using I-node routines: found 15 nodes, limit used is 5 STEP 1 (Newton iterations: 3) ^C C:\cygwin\home\rabbit\pGFEM-test> ----- Original Message ----- From: Jed Brown To: PETSc users list Sent: Thursday, May 12, 2011 6:41 PM Subject: Re: [petsc-users] nonzero prescribed boundary condition On Thu, May 12, 2011 at 06:35, Tian(ICT) wrote: Without any option, SNES converges never calling FormJacobian, and the solution seems correct. What is the reason for that the SNES solver never calls FormJacobian (mf option was not used)? Please send the following: 1. The code where you configure the solver, everything from SNESCreate to SNESSolve 2. The complete command line that you run with. 3. Output when you add "-snes_view -options_table" to the command line -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 12 06:15:45 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 12 May 2011 13:15:45 +0200 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <1302D684D3644298A3F2528BDEE08857@rti458laptop> <32DB8456522B4FABB725933C113D66D1@rti458laptop> <03FE9F5E-B08C-4163-9209-15F4DCCB740F@mcs.anl.gov> <71C9B313096748028405BE22EBAF85CD@rti458laptop> Message-ID: On Thu, May 12, 2011 at 13:01, Tian(ICT) wrote: > C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_view -options_tableclear > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > SNES Object: > type: ls > line search variant: SNESLineSearchCubic > alpha=0.0001, maxstep=1e+008, steptol=1e-012 > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-008, absolute=1e-050, solution=1e-008 > total number of linear solver iterations=3 > total number of function evaluations=4 > KSP Object: > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonaliza > tion with no iterative refinement > GMRES: happy breakdown tolerance 1e-030 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-005, absolute=1e-050, divergence=10000 > left preconditioning > PC Object: > type: ilu > ILU: 0 levels of fill > ILU: factor fill ratio allocated 1 > ILU: tolerance for zero pivot 1e-012 > out-of-place factorization > matrix ordering: natural > ILU: factor fill ratio needed 1 > Factored matrix follows > Matrix Object: > type=seqaij, rows=24, cols=24 > total: nonzeros=392, allocated nonzeros=840 > using I-node routines: found 15 nodes, limit used is 5 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=24, cols=24 > total: nonzeros=392, allocated nonzeros=1320 > using I-node routines: found 15 nodes, limit used is 5 This looks pretty clear that the matrix was assembled and the linear system converged in one iteration. Now to see how the matrix was assembled. Are you sure that your FormJacobian was not called? The option -snes_fd (maybe in an environment variable or a .petscrc file) could assemble a matrix, but it would log the number of function evaluations used to do that. I don't see any evidence that you are building a Jacobian using coloring. You can use a debugger to set a breakpoint in SNESComputeJacobian, then check the value of the pointer snes->ops->computejacobian. It should point at your FormJacobian. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu May 12 08:13:37 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 12 May 2011 08:13:37 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> On May 12, 2011, at 1:01 AM, Sylvain Barbot wrote: > Hi Barry, > > I have tested the direct solver with the shell matrix and it works > very well. In my code, the matrix-multiply operation is implemented > with the function "matrixantiplanefv". I am now trying to do it with > multi-grid. I am keeping a direct solver solely for the coarse-level. > I will do this coarse solve matrix-free with the function > "matrixantiplanecoarsefv". I am also setting up all the other > operations (residual, smoother, restriction, interpolation) with > matrix-free methods. The setup is done in the subroutine > "green_init_mg". > Yes, I understand all this and it sounds fine. > First I set up a PCMG preconditionner: > > CALL KSPCreate(PETSC_COMM_WORLD,c%ksp,ierr); > CALL KSPGetPC(c%ksp,c%pc,ierr); > CALL PCSetType(c%pc,PCMG,ierr); > CALL PCMGSetLevels(c%pc,c%nlevels,PETSC_NULL_OBJECT,ierr); > CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); > CALL PCMGSetCycleType(c%pc,PC_MG_CYCLE_V,ierr); > CALL PCMGSetNumberSmoothUp(c%pc,1,ierr); > CALL PCMGSetNumberSmoothDown(c%pc,1,ierr); > > > Then I set up the coarse solver: > > CALL MatCreateShell(PETSC_COMM_WORLD, & > > cinfo%zm*cinfo%ym*cinfo%xm*info%dof,cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof, > & > > cinfo%mz*cinfo%my*cinfo%mx*info%dof,cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof, > & > PETSC_NULL,c%cA,ierr); > CALL MatShellSetOperation(c%cA,MATOP_MULT,matrixantiplanecoarsefv,ierr); > CALL MatAssemblyBegin(c%cA,MAT_FINAL_ASSEMBLY,ierr); > CALL MatAssemblyEnd(c%cA,MAT_FINAL_ASSEMBLY,ierr); > CALL KSPSetOperators(c%cksp,c%cA,c%cA,DIFFERENT_NONZERO_PATTERN,ierr); > CALL KSPSetUp(c%cksp,ierr); > CALL PCMGGetCoarseSolve(c%pc,c%cksp,ierr); > > > Then I set up the interpolation and restriction: > > DO l=1,c%nlevels-1 > ... > ! shell matrix MxN with M>N > CALL MatCreateShell(PETSC_COMM_WORLD, & > > cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof,minfo%zm*minfo%ym*minfo%xm*minfo%dof, > & > > cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof,minfo%mz*minfo%my*minfo%mx*minfo%dof, > & > PETSC_NULL,c%interp(l),ierr); > > CALL MatShellSetOperation(c%interp(l),MATOP_MULT,matrixinterp,ierr); > CALL MatAssemblyBegin(c%interp(l),MAT_FINAL_ASSEMBLY,ierr); > CALL MatAssemblyEnd(c%interp(l),MAT_FINAL_ASSEMBLY,ierr); > > ! shell matrix MxN with M CALL MatCreateShell(PETSC_COMM_WORLD, & > > minfo%zm*minfo%ym*minfo%xm*minfo%dof,cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof, > & > > minfo%mz*minfo%my*minfo%mx*minfo%dof,cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof, > & > PETSC_NULL,c%restrict(l),ierr); > > CALL MatShellSetOperation(c%restrict(l),MATOP_MULT,matrixrestrict,ierr); > CALL MatAssemblyBegin(c%restrict(l),MAT_FINAL_ASSEMBLY,ierr); > CALL MatAssemblyEnd(c%restrict(l),MAT_FINAL_ASSEMBLY,ierr); > > ! define the intergrid transfer operations > CALL PCMGSetInterpolation(c%pc,l,c%interp(l),ierr); > CALL PCMGSetRestriction(c%pc,l,c%restrict(l),ierr); > > END DO > > Then i provide the smoother for each level: > > DO l=0,c%nlevels-1 > ! global size at current level > cinfo%mx=(info%mx/2**(c%nlevels-l))*2+1 > cinfo%my=(info%my/2**(c%nlevels-l))*2+1 > cinfo%mz=1 > ! locally-owned size at current level > CALL getmxl(x,info%xm,c%nlevels-l-1,cinfo%xm) > CALL getmxl(y,info%ym,c%nlevels-l-1,cinfo%ym) > cinfo%zm=1 > cinfo%dof=info%dof > > !----------------------------------------------------------------------------- > ! smoothing matrix > !----------------------------------------------------------------------------- > > ! square smoothing matrix > CALL MatCreateShell(PETSC_COMM_WORLD, & > > cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof,cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof, > & > > cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof,cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof, > & > PETSC_NULL,c%sA(l+1),ierr); > > CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. > CALL MatAssemblyBegin(c%sA(l+1),MAT_FINAL_ASSEMBLY,ierr); > CALL MatAssemblyEnd(c%sA(l+1),MAT_FINAL_ASSEMBLY,ierr); > > CALL PCMGGetSmoother(c%pc,l,sksp,ierr); > CALL KSPCreate(PETSC_COMM_WORLD,sksp,ierr); > CALL KSPSetOperators(sksp,c%sA(l+1),c%sA(l+1),DIFFERENT_NONZERO_PATTERN,ierr); > CALL KSPSetUp(sksp,ierr); > > !----------------------------------------------------------------------------- > ! residual operator > !----------------------------------------------------------------------------- > > ! square forward-model matrix > CALL MatCreateShell(PETSC_COMM_WORLD, & > > cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof,cinfo%zm*cinfo%ym*cinfo%xm*cinfo%dof, > & > > cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof,cinfo%mz*cinfo%my*cinfo%mx*cinfo%dof, > & > PETSC_NULL,c%rA(l+1),ierr); > > CALL MatShellSetOperation(c%rA(l+1),MATOP_MULT,matrixresidualfv,ierr); > CALL MatAssemblyBegin(c%rA(l+1),MAT_FINAL_ASSEMBLY,ierr); > CALL MatAssemblyEnd(c%rA(l+1),MAT_FINAL_ASSEMBLY,ierr); > > ! compute formula b - Ax with matrix-free matrix > CALL PCMGSetResidual(c%pc,l,matrixresidualfv,c%rA(l+1),ierr); > > END DO > > I also provide work space... > Despite that, I have a crash upon running KSPSetup() with "[0]PETSC > ERROR: PCSetUp_BJacobi()". It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. What solver options are you running with? > > > Also, I do no wish to provide a shell matrix for a direct solver at > the finest level but if I don't I get the error: > > MatGetVecs() line 7265 in src/mat/interface/matrix.c > [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c > [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: KSPSetUp() line > > So despite all my efforts, it looks that the solver still attempts to > do a direct solve. > This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. Barry > I would appreciate your comments. > Best wishes, > Sylvain > > 2011/5/10 Barry Smith : >> >> >> Barry >> >> >> On May 10, 2011, at 9:20 PM, Sylvain Barbot wrote: >> >>> Dear Jed, >>> >>> Thank you for your previous comments. I am still trying to design a >>> multigrid preconditionner for the Navier's equation of elasticity. I >>> have defined shell matrices for the interpolation, restriction, >>> smoothing, residual calculation and the fine-level solver (see >>> attached my_green.f90 file with stub functions). I thought that all >>> these shell matrices would be enough to do the job, but upon running >>> KSPSetup(), I get the following error: >>> >>> [0]PETSC ERROR: No support for this operation for this object type! >>> [0]PETSC ERROR: This matrix type, shell, does not support getting >>> diagonal block! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 >>> 14:42:02 CDT 2010 >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c >>> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c >>> >>> regarding your previous comments: "For Jacobi, you can have your >>> MatShell implement MatGetDiagonal and it will work. (...) If you want >>> to use a multiplicative relaxation like SOR, you would have to >>> implement it yourself.", I actually want to use SOR with my smoother >>> shell matrix. What am I doing wrong to make petsc still believe I want >>> to use Jacobi? >>> >> >> By default, it is using block Jacobi on each level (except the coarsest). You need to tell it to use the relaxation you have coded. For example if your shell matrix provides a SOR/relaxation method then you need to use -mg_levels_pc_type sor and it will call that on each level instead of block Jacobi. In looking at your code I see the only operation your provide your shell matrices are matrix-vector multiply. Therefore you cannot use SOR/relaxation. The only think you can use is GMRES (or some other Krylov method) with no preconditioner as the smoother. For this you can use -mg_levels_pc_type none -mg_levels_ksp_type gmres >> >> In your message you say "I actually want to use SOR with my smoother" to do this you need to provide the smoother function and set it with MatShellSetOperations(). >> >> >>> another related question: with a direct solver, one needs to assign a >>> shell matrix with a KSP object (KSPSetOperators); is it still required >>> when using multigrid? if I don't do so, I get the following error: >>> >> Where are you using the direct solver for, on the entire linear system or on the coarse mesh? In either case you NEED to provide an explicit (not shell) sparse matrix otherwise a direct solver cannot be used. From the message below it looks like you did not provide a matrix somewhere. If you don't provide a matrix a direct solver cannot be used. >> >> >> Barry >> >>> [0]PETSC ERROR: Null argument, when expecting valid pointer! >>> [0]PETSC ERROR: Null Object: Parameter # 1! >>> [0]PETSC ERROR: MatGetVecs() line 7265 in src/mat/interface/matrix.c >>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>> [0]PETSC ERROR: KSPSetUp() line 199 in src/ksp/ksp/interface/itfunc.c >>> >>> however, I don't understand why I would need this matrix in a >>> multigrid framework. >>> >>> Is there any example out there of using PCMG with shell matrices? >>> >>> Thanks in advance for your response. >>> Best wishes, >>> Sylvain Barbot >>> >>> >>> 2010/10/7 Jed Brown : >>>> On Thu, Oct 7, 2010 at 11:45, Sylvain Barbot >>>> wrote: >>>>> >>>>> questions: >>>>> 1) is it at all possible to specify this mode of operation, from the >>>>> finest to the coarser level, and back? Any examples out there? >>>> >>>> -pc_mg_type multiplicative, or >>>> PCMGSetType(pc,PC_MG_MULTIPLICATIVE); >>>> >>>>> >>>>> 2) is it readily possible to use matrix shells with DMMG? I imagine >>>>> the Jacobian matrix may simply be provided as a matrix shell. Is there >>>>> any examples of multi-grid methods with shell matrices online? >>>> >>>> You can do this, but you will have to define a custom smoother. For Jacobi, >>>> you can have your MatShell implement MatGetDiagonal and it will work. I >>>> thought you could implement MatGetDiagonalBlock for PBJacobi, but it's not >>>> currently written that way (though it should be and that would be an easy >>>> change to make). If you want to use a multiplicative relaxation like SOR, >>>> you would have to implement it yourself. If you need something like ILU for >>>> a smoother, then you will have to pay for a matrix. Note that one >>>> possibility is to assemble a matrix on all but the finest level so you can >>>> use stronger smoothers there, then make do with Jacobi on the finest level. >>>> >>>>> >>>>> 3) to deal with non-uniform sampling: can I provide the coordinates of >>>>> the finest grid with DASetCoordinates, then expect DMMG to provide the >>>>> subsampled coordinates at the coarser levels? >>>> >>>> Currently no, you have to set them on each level. Perhaps you could do this >>>> by rolling a loop over levels and applying MatRestrict using the restriction >>>> matrix from DMMG (this might not be the sampling that you want). >>>> Jed >>> >> >> From uerland at gmail.com Thu May 12 08:41:19 2011 From: uerland at gmail.com (Henning Sauerland) Date: Thu, 12 May 2011 15:41:19 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code Message-ID: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> Hi, I'm using PETSC with a finite element incompressible two-phase Navier-Stokes code. In 3D I'm currently struggling with the parallel performance. The domain partitioning is done with ParMeTis and PETSC is used for matrix assembly and solving. Unfortunately, for this problem the number of processors has a large influence on the number of iterations leading to a poor scaling. I have already tested a lot of solver-preconditioner combinations. LGMRES with ASM preconditioning seems to perform best. Applying -sub_pc_type lu helped a lot in 2D, but in 3D apart from reducing the number of iterations the whole solution takes more than 10 times longer. I attached the log_summary output for a problem with about 240000 unkowns (1 time step) using 4, 8 and 16 Intel Xeon E5450 processors (InfiniBand-connected). As far as I see the number of iterations seems to be the major issue here or am I missing something? I would appreciate any suggestions as to what I could try to improve the scaling. Thanks Henning -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 4pe.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 8pe.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 16pe.txt URL: -------------- next part -------------- From s4245690 at mailbox.tu-dresden.de Thu May 12 03:48:16 2011 From: s4245690 at mailbox.tu-dresden.de (martin flehmig) Date: Thu, 12 May 2011 10:48:16 +0200 Subject: [petsc-users] Matrix with different number of rows on different processes Message-ID: <1305190096.7527.6.camel@MacbookPro> Hi, is it possible to dictate how many rows a process owns on a matrix? For example: A is a square matrix with 10 rows/columns. There are two processes. cpu0 should own rows 0-7 and cpu1 the others. Thanks, martin From jed at 59A2.org Thu May 12 09:02:38 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 12 May 2011 16:02:38 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> Message-ID: On Thu, May 12, 2011 at 15:41, Henning Sauerland wrote: > Applying -sub_pc_type lu helped a lot in 2D, but in 3D apart from reducing > the number of iterations the whole solution takes more than 10 times longer. > Does -sub_pc_type ilu -sub_pc_factor_levels 2 (default is 0) help relative to the default? Direct subdomain solves in 3D are very expensive. How much does the system change between time steps? What "CFD" formulation is this (physics, discretization) and what regime (Reynolds and Mach numbers, etc)? I attached the log_summary output for a problem with about 240000 unkowns (1 > time step) using 4, 8 and 16 Intel Xeon E5450 processors > (InfiniBand-connected). As far as I see the number of iterations seems to be > the major issue here or am I missing something? Needing more iterations is the algorithmic part of the problem, but the relative cost of orthogonaliztaion is going up. You may want to see if the iteration count can stay reasonable with -ksp_type ibcgs. If this works algorithmically, it may ease the pain. Beyond that, the algorithmic scaling needs to be improved. How does the iteration count scale if you use a direct solver? (I acknowledge that it is not practical, but it provides some insight towards the underlying problem.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 12 09:03:26 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 12 May 2011 16:03:26 +0200 Subject: [petsc-users] Matrix with different number of rows on different processes In-Reply-To: <1305190096.7527.6.camel@MacbookPro> References: <1305190096.7527.6.camel@MacbookPro> Message-ID: On Thu, May 12, 2011 at 10:48, martin flehmig < s4245690 at mailbox.tu-dresden.de> wrote: > is it possible to dictate how many rows a process owns on a matrix? > > For example: > A is a square matrix with 10 rows/columns. There are two processes. cpu0 > should own rows 0-7 and cpu1 the others. > See the "m" and "n" arguments: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Mat/MatSetSizes.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From s_g at berkeley.edu Thu May 12 09:51:59 2011 From: s_g at berkeley.edu (Sanjay Govindjee) Date: Thu, 12 May 2011 07:51:59 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: <4DCBF40F.1020301@berkeley.edu> Sylvain, Is there a reason you are not using Prometheus (--download_prometheus=1) for your MG preconditioner? It was designed with 3D solid mechanics in mind. -sanjay On 5/11/11 11:12 PM, Sylvain Barbot wrote: > Dear Jed, > > During my recent visit to ETH, I talked at length about multi-grid > with Dave May who warned me about the issues of large > coefficient-contrasts. Most of my problems of interest for > tectonophysics and earthquake simulations are cases of relatively > smooth variations in elastic moduli. So I am not too worried about > this aspect of the problem. I appreciate your advice about trying > simpler solutions first. I have tested at length direct solvers of 2-D > and 3-D problems of elastic deformation and I am quite happy with the > results. My primary concern now is computation speed, especially for > 3-D problems, where i have of the order 512^3 degrees of freedom. I > was planning to test Jacobi and SOR smoothers. Is there another > smoother you recommend for this kind of problem? > > Thanks, > Sylvain > > 2011/5/11 Jed Brown: >> On Wed, May 11, 2011 at 04:20, Sylvain Barbot >> wrote: >>> I am still trying to design a >>> multigrid preconditionner for the Navier's equation of elasticity. >> I have heard, through an external source, that you have large jumps in both >> Young's modulus and Poisson ratio that are not grid aligned, including >> perhaps thin structures that span a large part of the domain. Such problems >> are pretty hard, so I suggest you focus on robustness and do not worry about >> low-memory implementation at this point. That is, you should assemble the >> matrices in a usual PETSc format instead of using MatShell to do everything >> matrix-free. This gives you access to much stronger smoothers. >> After you find a scheme that is robust enough for your purposes, _then_ you >> can make it low-memory by replacing some assembled matrices by MatShell. To >> realize most of the possible memory savings, it should be sufficient to do >> this on the finest level only. From danesh.daroui at ltu.se Thu May 12 10:33:48 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Thu, 12 May 2011 17:33:48 +0200 Subject: [petsc-users] Sparse Approximate Inverse (SPAI) preconditioner Message-ID: <1305214428.3340.6.camel@linux-sclt.site> Hi, I am using PETSc to solve a linear system of complex double precision data type. I intend to use SPAI preconditioner to solve the equation and I know that SPAI is not supported with complex systems. Since I have imaginary values only on diagonal, so according to my tests, approximate inverse of only real part of the coefficient matrix is a good preconditioner for my need. Therefore, I would like to install SPAI with PETSc and use it just to calculate approximate inverse of the real part of the matrix which should be possible. The problem is that PETSc can be configured to install SPAI when complex solver is being installed or I don't know how! The other questions that, is there any way to store the complex matrix in PETSc (like I do) and pass only real part of it to the routine which calculates approximate inverse? Otherwise I need to store both complex and real values which will eat up the memory. Regards, Danesh From bsmith at mcs.anl.gov Thu May 12 12:50:24 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 12 May 2011 12:50:24 -0500 Subject: [petsc-users] Sparse Approximate Inverse (SPAI) preconditioner In-Reply-To: <1305214428.3340.6.camel@linux-sclt.site> References: <1305214428.3340.6.camel@linux-sclt.site> Message-ID: <2B4F8572-1B0A-4466-B7B1-4C7A41E1423C@mcs.anl.gov> On May 12, 2011, at 10:33 AM, Danesh Daroui wrote: > > Hi, > > I am using PETSc to solve a linear system of complex double precision > data type. I intend to use SPAI preconditioner to solve the equation and > I know that SPAI is not supported with complex systems. Since I have > imaginary values only on diagonal, so according to my tests, approximate > inverse of only real part of the coefficient matrix is a good > preconditioner for my need. Therefore, I would like to install SPAI with > PETSc and use it just to calculate approximate inverse of the real part > of the matrix which should be possible. The problem is that PETSc can be > configured to install SPAI when complex solver is being installed or I > don't know how! The other questions that, is there any way to store the > complex matrix in PETSc (like I do) and pass only real part of it to the > routine which calculates approximate inverse? Otherwise I need to store > both complex and real values which will eat up the memory. No. PETSc either works with all objects as real. Or all objects as complex. There is no way to do some computations with real and others with complex. Barry > > Regards, > > Danesh > > From bsmith at mcs.anl.gov Thu May 12 14:54:29 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 12 May 2011 14:54:29 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> Message-ID: On May 12, 2011, at 8:41 AM, Henning Sauerland wrote: > Hi, > > I'm using PETSC with a finite element incompressible two-phase Navier-Stokes code. In 3D I'm currently struggling with the parallel performance. The domain partitioning is done with ParMeTis and PETSC is used for matrix assembly and solving. Unfortunately, for this problem the number of processors has a large influence on the number of iterations leading to a poor scaling. I have already tested a lot of solver-preconditioner combinations. LGMRES with ASM preconditioning seems to perform best. Applying -sub_pc_type lu helped a lot in 2D, but in 3D apart from reducing the number of iterations Exactly how much did it reduce the number of iterations? For example on 4 processes it is taking 500 iterations, with LU instead how many iterations does it take? 50? 100? 400? Also how many iterations does ILU need on one process? We have to get that iteration count down dramatically, to do this we need to start by understanding relatively how much the ILU is hurting you versus the overlapping Schwarz decomposition. Do you some kind of saddle-pioint thingy in your Jacobian? (if so I'm guessing you are not doing anything to deal with it). Barry > the whole solution takes more than 10 times longer. I attached the log_summary output for a problem with about 240000 unkowns (1 time step) using 4, 8 and 16 Intel Xeon E5450 processors (InfiniBand-connected). As far as I see the number of iterations seems to be the major issue here or am I missing something? I would appreciate any suggestions as to what I could try to improve the scaling. > > > Thanks > Henning > > > <4pe.txt><8pe.txt><16pe.txt> > > From gdiso at ustc.edu Thu May 12 22:13:21 2011 From: gdiso at ustc.edu (Gong Ding) Date: Fri, 13 May 2011 11:13:21 +0800 (CST) Subject: [petsc-users] Document mistake of SNESLineSearchSetPostCheck Message-ID: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> In the document of SNESLineSearchSetPostCheck, w = x + y However, the code implementation is w = x - y From bsmith at mcs.anl.gov Thu May 12 22:28:48 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 12 May 2011 22:28:48 -0500 Subject: [petsc-users] Document mistake of SNESLineSearchSetPostCheck In-Reply-To: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> References: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> Message-ID: Thanks I've updated the doc in petsc-dev Barry On May 12, 2011, at 10:13 PM, Gong Ding wrote: > n the document of SNESLineSearchSetPostCheck, > w = x + y > However, the code implementation is > w = x - y From rongtian at ncic.ac.cn Thu May 12 23:51:41 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Fri, 13 May 2011 12:51:41 +0800 Subject: [petsc-users] SNES solver In-Reply-To: References: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> Message-ID: What does this output really mean? Why does a converged Newton iteration still lead to a diverged SNES solve? C:\cygwin\home\rabbit\pGFEM-test>pgfem -ksp_monitor atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 KSP Residual norm 1.526322665926e-001 1 KSP Residual norm 1.664108108313e-003 2 KSP Residual norm 9.234526593231e-006 3 KSP Residual norm 7.035003988413e-008 0 KSP Residual norm 1.372225675693e-001 1 KSP Residual norm 1.955889208728e-003 2 KSP Residual norm 6.925412512315e-006 3 KSP Residual norm 1.652845110783e-008 0 KSP Residual norm 1.233183603661e-001 1 KSP Residual norm 1.751093239951e-003 2 KSP Residual norm 6.124066396591e-006 3 KSP Residual norm 1.431288831469e-008 C:\cygwin\home\rabbit\pGFEM-test>pgfem -ksp_monitor -snes_monitor atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 1.000000000000e+002 0 KSP Residual norm 1.526322665926e-001 1 KSP Residual norm 1.664108108313e-003 2 KSP Residual norm 9.234526593231e-006 3 KSP Residual norm 7.035003988413e-008 1 SNES Function norm 9.112654908737e+001 0 KSP Residual norm 1.372225675693e-001 1 KSP Residual norm 1.955889208728e-003 2 KSP Residual norm 6.925412512315e-006 3 KSP Residual norm 1.652845110783e-008 2 SNES Function norm 8.914492398201e+001 0 KSP Residual norm 1.233183603661e-001 1 KSP Residual norm 1.751093239951e-003 2 KSP Residual norm 6.124066396591e-006 3 KSP Residual norm 1.431288831469e-008 3 SNES Function norm 8.937681381791e+001 STEP 0 (Newton iterations: 3) diverged reason: -6 --Rong From jed at 59A2.org Fri May 13 03:37:20 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 13 May 2011 10:37:20 +0200 Subject: [petsc-users] SNES solver In-Reply-To: References: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> Message-ID: On Fri, May 13, 2011 at 06:51, Tian(ICT) wrote: > What does this output really mean? > Why does a converged Newton iteration still lead to a diverged SNES solve? > > Newton did not converge. > > STEP 0 (Newton iterations: 3) > > diverged reason: -6 > You should consider using PetscPrintf(comm,"%s\n",SNESConvergedReasons[reason]); instead of printing the numeric value. Or just run with -snes_converged_reason to have PETSc print it for you before SNESSolve returns. The reason is SNES_DIVERGED_LINE_SEARCH -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri May 13 07:36:27 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 13 May 2011 07:36:27 -0500 Subject: [petsc-users] SNES solver In-Reply-To: References: <26722737.21681305256401888.JavaMail.coremail@mail.ustc.edu> Message-ID: On May 13, 2011, at 3:37 AM, Jed Brown wrote: > On Fri, May 13, 2011 at 06:51, Tian(ICT) wrote: > What does this output really mean? > Why does a converged Newton iteration still lead to a diverged SNES solve? > > > Newton did not converge. > > > STEP 0 (Newton iterations: 3) > > diverged reason: -6 > > You should consider using > > PetscPrintf(comm,"%s\n",SNESConvergedReasons[reason]); > > instead of printing the numeric value. Or just run with -snes_converged_reason to have PETSc print it for you before SNESSolve returns. The reason is SNES_DIVERGED_LINE_SEARCH See http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#newton for hints on why SNES is not converging. Barry From uerland at gmail.com Fri May 13 08:50:43 2011 From: uerland at gmail.com (Henning Sauerland) Date: Fri, 13 May 2011 15:50:43 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> Message-ID: <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Let me first of all explain the problem I'm considering a bit more detailed. I'm working on two-phase flow problems in the low Reynolds number regime (laminar flow). The flow field is described by the incompressible Navier-Stokes equations and the phase interface is tracked implicitly using the level-set method. This leads to a strongly coupled problem of the flow field and the level-set field. That is, during one time step the Navier-Stokes equations are solved in a series of Picard iterations and subsequently the interface (level-set field) is advected in the flow field. Those 'two' steps are carried out until the fluid and the level-set field are converged. A typical output of my current testcase for one time step looks like that (showing the relative norm of the solution vector and the number of solver iterations): KSP Iterations: 170 Picard iteration step 1: 1.000000e+00 KSP Iterations: 151 Picard iteration step 2: 6.972740e-07 KSP Iterations: 4 Level-set iteration step 1: 2.619094e-06 KSP Iterations: 166 Picard iteration step 1: 1.124124e-06 KSP Iterations: 4 Level-set iteration step 2: 5.252072e-11 Time step 1 of 1, time: 0.005000 Excuse me for not mentioning it in the first place. The log_summary output on it's own may be misleading. For comparison I think one should probably concentrate on the iteration counts for one Picard iteration only. The problem is discretized using FEM (more precisely XFEM) with stabilized, trilinear hexahedral elements. As the XFEM approximation space is time-dependant as well as the physical properties at the nodes the resulting system may change quite significantly between time steps. Furthermore, the system matrix tends to be ill-conditioned which can luckily be greatly imporved using a diagonal scaling. On 12.05.2011, at 16:02, Jed Brown wrote: > On Thu, May 12, 2011 at 15:41, Henning Sauerland wrote: > Applying -sub_pc_type lu helped a lot in 2D, but in 3D apart from reducing the number of iterations the whole solution takes more than 10 times longer. > > Does -sub_pc_type ilu -sub_pc_factor_levels 2 (default is 0) help relative to the default? Direct subdomain solves in 3D are very expensive. How much does the system change between time steps? ILU(2) requires less than half the number of KSP iterations, but it scales similar to ILU(0) and requires about 1/3 more time. > > What "CFD" formulation is this (physics, discretization) and what regime (Reynolds and Mach numbers, etc)? > > I attached the log_summary output for a problem with about 240000 unkowns (1 time step) using 4, 8 and 16 Intel Xeon E5450 processors (InfiniBand-connected). As far as I see the number of iterations seems to be the major issue here or am I missing something? > > Needing more iterations is the algorithmic part of the problem, I guess you are talking about the nonlinear iterations? I was always referring to the KSP iterations and I thought that the ksp iteration count grows with increasing number of processors is more or less solely related to the iterative solver and preconditioner. > but the relative cost of orthogonaliztaion is going up. You may want to see if the iteration count can stay reasonable with -ksp_type ibcgs. If this works algorithmically, it may ease the pain. Beyond that, the algorithmic scaling needs to be improved. How does the iteration count scale if you use a direct solver? (I acknowledge that it is not practical, but it provides some insight towards the underlying problem.) ibcgs is slightly faster, requiring less number of ksp iterations compared to lgmres. Unfortunately, the iteration count scales very similar to lgmres and generally the lack of robustness of bcgs solvers turns out to problematic for tougher testcases in my experience. Thanks Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilyascfd at gmail.com Fri May 13 08:56:03 2011 From: ilyascfd at gmail.com (ilyas ilyas) Date: Fri, 13 May 2011 16:56:03 +0300 Subject: [petsc-users] -log_summary output Message-ID: Hello, Would you please help me to understand two attached -log_summary outputs. Both are from same cfd-petsc code, taken at different times at the end of first time-step. Domain : 256^3, periodic (defined in DA's) Ncores : 256 After a couple of hundered iterations, one or more cores killed/dumped and my code exits.This is not an issue with a coarse resolution (say 32^3,64^3) May this be related to my code? (Assuming enough memory available in the system). What is wrong with MatXXX ? Thanks a lot, Ilyas. Energy Inst. ITU -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 185432.out Type: application/octet-stream Size: 55573 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 185433.out Type: application/octet-stream Size: 55879 bytes Desc: not available URL: From jed at 59A2.org Fri May 13 09:34:50 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 13 May 2011 16:34:50 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: How small is the Reynolds number? There is a difference between 0.1 and 100, although both may be laminar. On Fri, May 13, 2011 at 15:50, Henning Sauerland wrote: > The problem is discretized using FEM (more precisely XFEM) with stabilized, > trilinear hexahedral elements. As the XFEM approximation space is > time-dependant as well as the physical properties at the nodes the resulting > system may change quite significantly between time steps. > I assume the XFEM basis is just resolving the jump across the interface. Does this mean the size of the matrix is changing as the interface moves? In any case, it looks like you can't amortize setup costs between time steps, so we need a solution short of a direct solve. > ILU(2) requires less than half the number of KSP iterations, but it scales > similar to ILU(0) and requires about 1/3 more time. > Eventually, we're going to need a coarse level to battle the poor scaling with more processes. Hopefully that will alleviate the need for these more expensive local solves. > I guess you are talking about the nonlinear iterations? I was always > referring to the KSP iterations and I thought that the ksp iteration count > grows with increasing number of processors is more or less solely related to > the iterative solver and preconditioner. > I meant linear iterations. It is mostly dependent on preconditioner. The increased iteration count (when a direct subdomain solver is used, inexact subdomain solves confuse things) is likely due to the fundamental scaling for elliptic problems. There are many ways to improve constants, but you need a coarse level to fix the asymptotics. Unfortunately, multigrid methods for XFEM are a recent topic. Perhaps the best results I have seen (at conferences) use some geometric information in an otherwise algebraic framework. For this problem (unlike many fracture problems), the influence of the extended basis functions may be local enough that you can build a coarse level using the conventional part of the problem. The first thing I would try is probably to see if a direct solve with the conventional part makes an effective coarse level. If that works, I would see if ML or Hypre can do a reasonable job with that part of the problem. I have no great confidence that this will work, it's highly dependent on how local the influence of the extended basis functions is. Perhaps you have enough experience with the method to hypothesize. Note: for the conventional part of the problem, it is still incompressible flow. It sounds like you are using equal-order elements (Q1-Q1 stabilized; PSPG or Dohrmann&Bochev?). For those elements, you will want to interlace the velocity and pressure degrees of freedom, then use a smoother that respects the block size. PETSc's ML and Hypre interfaces forward this information if you set the block size on the matrix. When using ML, you'll probably have to make the smoothers stronger. There are also some "energy minimization" options that may help. > ibcgs is slightly faster, requiring less number of ksp iterations > compared to lgmres. Unfortunately, the iteration count scales very similar > to lgmres and generally the lack of robustness of bcgs solvers turns out to > problematic for tougher testcases in my experience. > Yes, that suggestion was only an attempt to improve the constants a little. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri May 13 10:16:18 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 13 May 2011 10:16:18 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: On May 13, 2011, at 9:34 AM, Jed Brown wrote: > How small is the Reynolds number? There is a difference between 0.1 and 100, although both may be laminar. > > On Fri, May 13, 2011 at 15:50, Henning Sauerland wrote: > The problem is discretized using FEM (more precisely XFEM) with stabilized, trilinear hexahedral elements. As the XFEM approximation space is time-dependant as well as the physical properties at the nodes the resulting system may change quite significantly between time steps. > > I assume the XFEM basis is just resolving the jump across the interface. Does this mean the size of the matrix is changing as the interface moves? In any case, it looks like you can't amortize setup costs between time steps, so we need a solution short of a direct solve. > > ILU(2) requires less than half the number of KSP iterations, but it scales similar to ILU(0) and requires about 1/3 more time. I think this is a more important point then you and Jed may give it credit for. It is not clear to me that the worse convergence (of the linear solve) with the number of subdomains is the chief issue you should be concerned about. An equally important issue is that the ILU(0) is not a good preconditioner (and while user a more direct solver, LU or ILU(2)) helps the convergence it is too expensive). Are you using AIJ matrix (and hence it is point ILU?) see below > > Eventually, we're going to need a coarse level to battle the poor scaling with more processes. Hopefully that will alleviate the need for these more expensive local solves. > > I guess you are talking about the nonlinear iterations? I was always referring to the KSP iterations and I thought that the ksp iteration count grows with increasing number of processors is more or less solely related to the iterative solver and preconditioner. > > I meant linear iterations. It is mostly dependent on preconditioner. The increased iteration count (when a direct subdomain solver is used, inexact subdomain solves confuse things) is likely due to the fundamental scaling for elliptic problems. There are many ways to improve constants, but you need a coarse level to fix the asymptotics. > > Unfortunately, multigrid methods for XFEM are a recent topic. Perhaps the best results I have seen (at conferences) use some geometric information in an otherwise algebraic framework. For this problem (unlike many fracture problems), the influence of the extended basis functions may be local enough that you can build a coarse level using the conventional part of the problem. The first thing I would try is probably to see if a direct solve with the conventional part makes an effective coarse level. If that works, I would see if ML or Hypre can do a reasonable job with that part of the problem. > > I have no great confidence that this will work, it's highly dependent on how local the influence of the extended basis functions is. Perhaps you have enough experience with the method to hypothesize. > > Note: for the conventional part of the problem, it is still incompressible flow. It sounds like you are using equal-order elements (Q1-Q1 stabilized; PSPG or Dohrmann&Bochev?). For those elements, you will want to interlace the velocity and pressure degrees of freedom, then use a smoother that respects the block size. Is Jed is correct on this? That you have equal order elements hence all the degrees of freedom live at the same points, then you can switch to BAIJ matrix format and ILU becomes automatically point block ILU and it may work much better as Jed indicates. But you can make this change before mucking with coarse grid solves or multilevel methods. I'm guess that the point-block ILU will be a good bit better and since this is easily checked (just create a BAIJ matrix and set the block size appropriately and leave the rest of the code unchanged (well interlace the variables if they are not currently interlaced). Barry > PETSc's ML and Hypre interfaces forward this information if you set the block size on the matrix. When using ML, you'll probably have to make the smoothers stronger. There are also some "energy minimization" options that may help. > ibcgs is slightly faster, requiring less number of ksp iterations compared to lgmres. Unfortunately, the iteration count scales very similar to lgmres and generally the lack of robustness of bcgs solvers turns out to problematic for tougher testcases in my experience. > > Yes, that suggestion was only an attempt to improve the constants a little. From bsmith at mcs.anl.gov Fri May 13 10:27:54 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 13 May 2011 10:27:54 -0500 Subject: [petsc-users] -log_summary output In-Reply-To: References: Message-ID: On May 13, 2011, at 8:56 AM, ilyas ilyas wrote: > > Hello, > > Would you please help me to understand two attached -log_summary outputs. Both are from same cfd-petsc code, taken at different times at the end of first time-step. > > Domain : 256^3, periodic (defined in DA's) > Ncores : 256 > > After a couple of hundered iterations, one or more cores killed/dumped and my code exits.This is not an issue with a coarse resolution (say 32^3,64^3) May this be related to my code? Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Distributed array 1 0 0 0 Vec 597 579 340156096 0 Vec Scatter 3 1 868 0 Index Set 9 9 1700440 0 I'm thinking there may be a memory leak with the vectors. During the run you sent us it is creating 597 vectors but only destroying 579. If each time-step you are leaking vectors it will eventually run out of memory and stop. Check that all your Vec creations have a matching destroy. Also make sure that is true for matrices. You can run with the options -malloc -malloc_debug -malloc_dump as a test and it will print out all the memory that PETSc used that was not freed. Ideally this will always report nothing after a run. BTW: Don't use these options when running for performance, just use them to test for memory leaks. > (Assuming enough memory available in the system). What is wrong with MatXXX ? Not sure what you mean here? There is no MatXXX in the output you sent us. It is true that since the matrices take up a lot of memory when memory is leaked often it crashes when trying to allocate a matrix. Barry > > Thanks a lot, > > Ilyas. > Energy Inst. > ITU > <185432.out><185433.out> From bsmith at mcs.anl.gov Fri May 13 19:53:54 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 13 May 2011 19:53:54 -0500 Subject: [petsc-users] Fwd: SNES examples question References: Message-ID: Begin forwarded message: > From: Boyana Norris > Date: May 13, 2011 4:01:30 PM CDT > To: Barry Smith > Cc: Qian Zhu > Subject: SNES examples question > > Hi Barry, > > Do you know what the valid values for the various model parameters in snes ex20 and ex29 are? We are looking for something we can vary similar to how we change the lid velocity and grashof parameters in the driven cavity problems, in order to generate linear systems with different characteristics. > > Thank you! > Boyana > > -- > Boyana Norris, Ph.D., Computer Scientist, Argonne National Laboratory > norris at mcs.anl.gov, +1.630.252.7908, http://www.mcs.anl.gov/~norris/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilyascfd at gmail.com Sat May 14 06:48:17 2011 From: ilyascfd at gmail.com (ilyas ilyas) Date: Sat, 14 May 2011 14:48:17 +0300 Subject: [petsc-users] -log_summary output In-Reply-To: References: Message-ID: Thank you Barry, I will check my code for memory leak. Sorry for the question. I was not clear enough. I was trying to point out Time Ratios for MatView(in the first output) and MatView,MatGetRowIJ (in the second output). Why are these values huge compared to others? Regards, Ilyas 2011/5/13 Barry Smith > > On May 13, 2011, at 8:56 AM, ilyas ilyas wrote: > > > > > Hello, > > > > Would you please help me to understand two attached -log_summary outputs. > Both are from same cfd-petsc code, taken at different times at the end of > first time-step. > > > > Domain : 256^3, periodic (defined in DA's) > > Ncores : 256 > > > > After a couple of hundered iterations, one or more cores killed/dumped > and my code exits.This is not an issue with a coarse resolution (say > 32^3,64^3) May this be related to my code? > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Distributed array 1 0 0 0 > Vec 597 579 340156096 0 > Vec Scatter 3 1 868 0 > Index Set 9 9 1700440 0 > > I'm thinking there may be a memory leak with the vectors. During the run > you sent us it is creating 597 vectors but only destroying 579. If each > time-step you are leaking vectors it will eventually run out of memory and > stop. Check that all your Vec creations have a matching destroy. Also make > sure that is true for matrices. You can run with the options -malloc > -malloc_debug -malloc_dump as a test and it will print out all the memory > that PETSc used that was not freed. Ideally this will always report nothing > after a run. BTW: Don't use these options when running for performance, just > use them to test for memory leaks. > > > > (Assuming enough memory available in the system). What is wrong with > MatXXX ? > > Not sure what you mean here? There is no MatXXX in the output you sent > us. It is true that since the matrices take up a lot of memory when memory > is leaked often it crashes when trying to allocate a matrix. > > Barry > > > > > Thanks a lot, > > > > Ilyas. > > Energy Inst. > > ITU > > <185432.out><185433.out> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat May 14 07:42:35 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 14 May 2011 07:42:35 -0500 Subject: [petsc-users] -log_summary output In-Reply-To: References: Message-ID: On May 14, 2011, at 6:48 AM, ilyas ilyas wrote: > Thank you Barry, > I will check my code for memory leak. > > Sorry for the question. I was not clear enough. > I was trying to point out Time Ratios for MatView(in the first output) and MatView,MatGetRowIJ (in the second output). MatView() will always have a large time ratio. I cannot explain why MatGetRowIJ() has such a huge ratio, in general this routine takes so little of the compute time it doesn't matter. Barry > Why are these values huge compared to others? > > Regards, > Ilyas > > > 2011/5/13 Barry Smith > > On May 13, 2011, at 8:56 AM, ilyas ilyas wrote: > > > > > Hello, > > > > Would you please help me to understand two attached -log_summary outputs. Both are from same cfd-petsc code, taken at different times at the end of first time-step. > > > > Domain : 256^3, periodic (defined in DA's) > > Ncores : 256 > > > > After a couple of hundered iterations, one or more cores killed/dumped and my code exits.This is not an issue with a coarse resolution (say 32^3,64^3) May this be related to my code? > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Distributed array 1 0 0 0 > Vec 597 579 340156096 0 > Vec Scatter 3 1 868 0 > Index Set 9 9 1700440 0 > > I'm thinking there may be a memory leak with the vectors. During the run you sent us it is creating 597 vectors but only destroying 579. If each time-step you are leaking vectors it will eventually run out of memory and stop. Check that all your Vec creations have a matching destroy. Also make sure that is true for matrices. You can run with the options -malloc -malloc_debug -malloc_dump as a test and it will print out all the memory that PETSc used that was not freed. Ideally this will always report nothing after a run. BTW: Don't use these options when running for performance, just use them to test for memory leaks. > > > > (Assuming enough memory available in the system). What is wrong with MatXXX ? > > Not sure what you mean here? There is no MatXXX in the output you sent us. It is true that since the matrices take up a lot of memory when memory is leaked often it crashes when trying to allocate a matrix. > > Barry > > > > > Thanks a lot, > > > > Ilyas. > > Energy Inst. > > ITU > > <185432.out><185433.out> > > From jed at 59A2.org Sat May 14 13:14:29 2011 From: jed at 59A2.org (Jed Brown) Date: Sat, 14 May 2011 20:14:29 +0200 Subject: [petsc-users] Fwd: SNES examples question In-Reply-To: References: Message-ID: On Sat, May 14, 2011 at 02:53, Barry Smith wrote: > Do you know what the valid values for the various model parameters in snes > ex20 and ex29 are? We are looking for something we can vary similar to how > we change the lid velocity and grashof parameters in the driven cavity > problems, in order to generate linear systems with different > characteristics. The options are given in the help strings. Do you want the names of the non-dimensional parameters? Do you want to know how various parameters affect the system? In the case of ex20, the exponent beta is the only free parameter. If temperature is always positive, I believe there are no well-posedness issues for any beta. Positive beta becomes degenerate as T -> 0, negative beta becomes singular in that limit. It appears that the method works for all positive beta. When beta is negative, this discretization (or maybe just the line search) is not monotone so it may compute negative temperatures (program crashes). Negative beta is physical for other heat transport problems. I can't give a similarly detailed explanation of ex29, but low viscosity and low resistivity should tend to make it less diffusive (closer to ideal MHD which is hyperbolic). This discretization seems to produce singular linear systems. -------------- next part -------------- An HTML attachment was scrubbed... URL: From norris at mcs.anl.gov Sat May 14 13:32:49 2011 From: norris at mcs.anl.gov (Boyana Norris) Date: Sat, 14 May 2011 13:32:49 -0500 Subject: [petsc-users] Fwd: SNES examples question In-Reply-To: References: Message-ID: Hi Jed, I'm mainly interested in the parameter limits -- I know how to specify them, but I don't see any explanation of valid values in the code. So for ex20, how large can beta get and still be meaningful? For ex29, are you saying that it's always generating a singular linear system, or only for low viscosity and resistivity? Thanks! Boyana -- Boyana Norris, Ph.D., Computer Scientist, Argonne National Laboratory norris at mcs.anl.gov, +1.630.252.7908, http://www.mcs.anl.gov/~norris/ On May 14, 2011, at 1:14 PM, Jed Brown wrote: > On Sat, May 14, 2011 at 02:53, Barry Smith wrote: > Do you know what the valid values for the various model parameters in snes ex20 and ex29 are? We are looking for something we can vary similar to how we change the lid velocity and grashof parameters in the driven cavity problems, in order to generate linear systems with different characteristics. > > The options are given in the help strings. Do you want the names of the non-dimensional parameters? Do you want to know how various parameters affect the system? > > In the case of ex20, the exponent beta is the only free parameter. If temperature is always positive, I believe there are no well-posedness issues for any beta. Positive beta becomes degenerate as T -> 0, negative beta becomes singular in that limit. It appears that the method works for all positive beta. When beta is negative, this discretization (or maybe just the line search) is not monotone so it may compute negative temperatures (program crashes). Negative beta is physical for other heat transport problems. > > I can't give a similarly detailed explanation of ex29, but low viscosity and low resistivity should tend to make it less diffusive (closer to ideal MHD which is hyperbolic). This discretization seems to produce singular linear systems. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sat May 14 14:19:35 2011 From: jed at 59A2.org (Jed Brown) Date: Sat, 14 May 2011 21:19:35 +0200 Subject: [petsc-users] Fwd: SNES examples question In-Reply-To: References: Message-ID: On Sat, May 14, 2011 at 20:32, Boyana Norris wrote: > I'm mainly interested in the parameter limits -- I know how to specify > them, but I don't see any explanation of valid values in the code. So for > ex20, how large can beta get and still be meaningful? > As a model problem or as "real" radiation-diffusion? I've seen at least "3.5" in papers, but higher may be possible. I put "3.5" in quotes because that paper also had a flux limiter and an ionization model so the expression was a bit different. As a model problem, I think you can take it arbitrarily large as long as the boundary conditions are such that it is always bounded below by a positive constant (and Newton doesn't send it off to a negative temperature). It's worth considering the Newton linearization for this problem. The flux is - T^beta \grad T so the linearization applied to an increment dT is - T^beta grad dT - (beta T^{beta-1} grad T) dT which looks like isotropic diffusion with diffusivity T^beta plus advection with "velocity" - beta T^{beta-1} grad T . You can estimate the cell Peclet number as Pe_h = h beta T^{beta-1} |grad T| / T^beta = h beta |grad T| / T . In general, larger values of beta will produce thinner boundary layers at the right edge of the domain where T is small and grad T is large. Due to entropy considerations, heat still flows from the warm side to the cold side, thus advection never dominates diffusion at a converged state (but it could, depending on the initial guess, at some earlier point in the iteration). If you are just interested in the linear systems, this is a pretty indirect way to get variable-coefficient advection-diffusion. > > For ex29, are you saying that it's always generating a singular linear > system, or only for low viscosity and resistivity? > Actually, it looks like that was just ILU and non-pivoting LU producing singular preconditioners. With no preconditioner, KSP estimates reasonable condition numbers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From uerland at gmail.com Sun May 15 08:23:38 2011 From: uerland at gmail.com (Henning Sauerland) Date: Sun, 15 May 2011 15:23:38 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: <9CC39E7A-989A-4F53-987D-D6602D9AEC5F@gmail.com> > How small is the Reynolds number? There is a difference between 0.1 and 100, although both may be laminar. It's more like in the range of 100. > I assume the XFEM basis is just resolving the jump across the interface. Does this mean the size of the matrix is changing as the interface moves? In any case, it looks like you can't amortize setup costs between time steps, so we need a solution short of a direct solve. Yes, exactly the pressure approximation space is enriched with discontinuous functions. Due to the XFEM basis I then get an additional degree of freedom at the nodes of elements cut by the interface. > Unfortunately, multigrid methods for XFEM are a recent topic. Perhaps the best results I have seen (at conferences) use some geometric information in an otherwise algebraic framework. For this problem (unlike many fracture problems), the influence of the extended basis functions may be local enough that you can build a coarse level using the conventional part of the problem. The first thing I would try is probably to see if a direct solve with the conventional part makes an effective coarse level. If that works, I would see if ML or Hypre can do a reasonable job with that part of the problem. > > I have no great confidence that this will work, it's highly dependent on how local the influence of the extended basis functions is. Perhaps you have enough experience with the method to hypothesize. Unfortunately, I don't have any practical experience with multigrid methods, but I'm afraid that the conventional part won't make a good coarse level. Depending on the test case (large density ratios between the phases) the standard approximation won't do well. Anyway, I'll take a closer look at the multigrid topic. > Note: for the conventional part of the problem, it is still incompressible flow. It sounds like you are using equal-order elements (Q1-Q1 stabilized; PSPG or Dohrmann&Bochev?). Q1-Q1 SUPG, PSPG stabilized. > For those elements, you will want to interlace the velocity and pressure degrees of freedom, then use a smoother that respects the block size. PETSc's ML and Hypre interfaces forward this information if you set the block size on the matrix. When using ML, you'll probably have to make the smoothers stronger. There are also some "energy minimization" options that may help. Best, Henning From uerland at gmail.com Sun May 15 08:23:58 2011 From: uerland at gmail.com (Henning Sauerland) Date: Sun, 15 May 2011 15:23:58 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> > I think this is a more important point then you and Jed may give it credit for. It is not clear to me that the worse convergence (of the linear solve) with the number of subdomains is the chief issue you should be concerned about. An equally important issue is that the ILU(0) is not a good preconditioner (and while user a more direct solver, LU or ILU(2)) helps the convergence it is too expensive). Are you using AIJ matrix (and hence it is point ILU?) see below > Is Jed is correct on this? That you have equal order elements hence all the degrees of freedom live at the same points, then you can switch to BAIJ matrix format and ILU becomes automatically point block ILU and it may work much better as Jed indicates. But you can make this change before mucking with coarse grid solves or multilevel methods. I'm guess that the point-block ILU will be a good bit better and since this is easily checked (just create a BAIJ matrix and set the block size appropriately and leave the rest of the code unchanged (well interlace the variables if they are not currently interlaced). Unfortunately, it is not that easy to switch to BAIJ. I only have a local XFEM basis. That is, I have variable numbers of degrees of freedom throughout the domain (nodes of elements cut by the fluid interface have on additional "XFEM unknown"). At the moment, I don't know how or if I can handle the XFEM block when using BAIJ. Best, Henning From jed at 59A2.org Sun May 15 08:57:03 2011 From: jed at 59A2.org (Jed Brown) Date: Sun, 15 May 2011 15:57:03 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <9CC39E7A-989A-4F53-987D-D6602D9AEC5F@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <9CC39E7A-989A-4F53-987D-D6602D9AEC5F@gmail.com> Message-ID: On Sun, May 15, 2011 at 15:23, Henning Sauerland wrote: > Unfortunately, I don't have any practical experience with multigrid > methods, but I'm afraid that the conventional part won't make a good coarse > level. Depending on the test case (large density ratios between the phases) > the standard approximation won't do well. Anyway, I'll take a closer look at > the multigrid topic. > Perhaps there is a homogenized problem for which the conventional method is a good approximation for long-range interaction? For example, if XFEM basis functions could be clipped and then eliminated locally. If you find a method in the literature that you would like to try, let us know and we can give advice for implementation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sun May 15 08:59:23 2011 From: jed at 59A2.org (Jed Brown) Date: Sun, 15 May 2011 15:59:23 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> Message-ID: On Sun, May 15, 2011 at 15:23, Henning Sauerland wrote: > Unfortunately, it is not that easy to switch to BAIJ. I only have a local > XFEM basis. That is, I have variable numbers of degrees of freedom > throughout the domain (nodes of elements cut by the fluid interface have on > additional "XFEM unknown"). At the moment, I don't know how or if I can > handle the XFEM block when using BAIJ. This is a problem, but you can still order unknowns in the vector so that velocity and pressure all come together, then AIJ/Inode will treat the components together which will tend to make ILU stronger. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun May 15 11:41:26 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 15 May 2011 11:41:26 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> Message-ID: <5B65C909-7B2F-45AC-8AAB-30F3B3F58979@mcs.anl.gov> On May 15, 2011, at 8:59 AM, Jed Brown wrote: > On Sun, May 15, 2011 at 15:23, Henning Sauerland wrote: > Unfortunately, it is not that easy to switch to BAIJ. I only have a local XFEM basis. That is, I have variable numbers of degrees of freedom throughout the domain (nodes of elements cut by the fluid interface have on additional "XFEM unknown"). At the moment, I don't know how or if I can handle the XFEM block when using BAIJ. > > This is a problem, but you can still order unknowns in the vector so that velocity and pressure all come together, then AIJ/Inode will treat the components together which will tend to make ILU stronger. This may help a bit but I'm not sure how dramatically. What we could really use is a point block ILU for AIJ matrices with inodes. That is each inode set of rows is eliminated simultaneously (by factoring the little diagonal block associated with that inode). This would possibly give the better convergence rate of point block ILU for this problem instead of point ILU. I don't think it is terribly difficult to code but it is a project and not something that could be whipped out in an hour. Hmm, maybe one of our summer students? Barry From jed at 59A2.org Sun May 15 11:47:03 2011 From: jed at 59A2.org (Jed Brown) Date: Sun, 15 May 2011 18:47:03 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <5B65C909-7B2F-45AC-8AAB-30F3B3F58979@mcs.anl.gov> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <5B65C909-7B2F-45AC-8AAB-30F3B3F58979@mcs.anl.gov> Message-ID: On Sun, May 15, 2011 at 18:41, Barry Smith wrote: > This may help a bit but I'm not sure how dramatically. What we could really > use is a point block ILU for AIJ matrices with inodes. That is each inode > set of rows is eliminated simultaneously (by factoring the little diagonal > block associated with that inode). But, pivoting aside, what is the difference between point and point-block ILU when there is no discarded fill within the block? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun May 15 11:53:22 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 15 May 2011 11:53:22 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <5B65C909-7B2F-45AC-8AAB-30F3B3F58979@mcs.anl.gov> Message-ID: <155BA6BC-0220-4D36-83C5-AF6E19DF60E0@mcs.anl.gov> On May 15, 2011, at 11:47 AM, Jed Brown wrote: > On Sun, May 15, 2011 at 18:41, Barry Smith wrote: > This may help a bit but I'm not sure how dramatically. What we could really use is a point block ILU for AIJ matrices with inodes. That is each inode set of rows is eliminated simultaneously (by factoring the little diagonal block associated with that inode). > > But, pivoting aside, what is the difference between point and point-block ILU when there is no discarded fill within the block? You are right, since the little diagonal blocks are dense by definition of inodes there is no difference. So you could be right, using an interlaced ordering (if it is not currently being used) could help the convergence, possibly a good deal. Worth trying. Barry From clemens.domanig at uibk.ac.at Mon May 16 01:28:07 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Mon, 16 May 2011 08:28:07 +0200 Subject: [petsc-users] VecSetValue. fails with -O1/2/3 Message-ID: <4DD0C3F7.5060504@uibk.ac.at> Hi everyone, maybe someone can explain to me what is going wrong with my code. In my programm I read values from a vector A and write them at new positions into a vector B. This works well as long as I don't use any optimization-flags when compiling the code. If I use -O1/2/3 the petsc tells me: "out of range error" at VecSetValue. It always happens at the some position although I'm doing this kind of operations at different places again. petsc-version 3.1.6 g++-version: gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) Thx for your help - C. Domanig From lizs at mail.uc.edu Mon May 16 01:36:57 2011 From: lizs at mail.uc.edu (Li, Zhisong (lizs)) Date: Mon, 16 May 2011 06:36:57 +0000 Subject: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? Message-ID: <88D7E3BB7E1960428303E760100374515F13A729@BL2PRD0103MB055.prod.exchangelabs.com> Hi, Petsc Team, In my knowledge of Petsc, the PCMG object is the main tool for general multigrid calculations, as DMMG only works for simple constant matrix and RHS. But it's difficult to find any example code for using PCMG. I checked out the petsc-user mail archive on this topic but still not find a good description on it. I encounter similar errors in the practice as some previous question-raisers did. Here is my simple code implementing the PCMG based on my understanding from the user manual and tutorial. It is supposed to have multigrid of only two levels (finest and coarsest grid). The error message indicates SEGV segmentation violation because of this part of code. Anything wrong with this implementation? DAGetMatrix(da, MATAIJ, &M); KSPCreate(PETSC_COMM_WORLD, &ksp); KSPSetType(ksp, KSPGMRES); KSPGetPC(ksp, &pcmg); PCSetType(pcmg, PCMG); PCMGSetLevels(pcmg, 2, &PETSC_COMM_WORLD); PCMGSetType(pcmg, PC_MG_MULTIPLICATIVE); PCMGSetCycleType(pcmg, PC_MG_CYCLE_W); PCMGSetCyclesOnLevel(pcmg, 0, 1); PCMGSetCyclesOnLevel(pcmg, 1, 1); PCMGGetCoarseSolve(pcmg, &ksp); PCMGGetSmoother(pcmg, 0, &ksp); PCMGGetSmoother(pcmg, 1, &ksp); PCMGSetInterpolation(pcmg, 1, M); PCMGSetRestriction(pcmg, 1, M); PCMGSetResidual(pcmg, 0, PCMGDefaultResidual, M); PCMGSetResidual(pcmg, 1, PCMGDefaultResidual, M); The mandatory remaining PCMGSetRhs( ), PCMGSetX( ) and PCMGSetR( ) functions should be generated by Petsc automatically. It is tough to learn some Petsc functions when no detailed example and few guidance is provided. I wish you can add some tutorials codes on PCMG usage in the future version of Petsc. Thank you very much. Zhisong Li -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 16 01:49:38 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 16 May 2011 08:49:38 +0200 Subject: [petsc-users] VecSetValue. fails with -O1/2/3 In-Reply-To: <4DD0C3F7.5060504@uibk.ac.at> References: <4DD0C3F7.5060504@uibk.ac.at> Message-ID: Try with valgrind. The compiler is probably initializing a variable only at -O0. On May 16, 2011 8:28 AM, "Clemens Domanig" wrote: Hi everyone, maybe someone can explain to me what is going wrong with my code. In my programm I read values from a vector A and write them at new positions into a vector B. This works well as long as I don't use any optimization-flags when compiling the code. If I use -O1/2/3 the petsc tells me: "out of range error" at VecSetValue. It always happens at the some position although I'm doing this kind of operations at different places again. petsc-version 3.1.6 g++-version: gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) Thx for your help - C. Domanig -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 07:27:29 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Mon, 16 May 2011 20:27:29 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at> Message-ID: <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> Hi, SNES does not converge. After I tested following FAQ hints but still could not find a reason. Could you please refer to the following THREE outputs to give me some hints. ================OUTPUT 1============= C:\cygwin\home\rabbit\pGFEM-test>pgfem -info [0] PetscInitialize(): PETSc successfully started: number of processors = 1 [0] PetscGetHostName(): Rejecting domainname, likely is NIS RTI458-LAPTOP=(?. [0] PetscInitialize(): Running on machine: RTI458-LAPTOP [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 -2080374784 max tags = 2147483647 [0] PetscCommDuplicate(): returning tag 2147483647 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850688 -20803747 84 [0] PetscCommDuplicate(): returning tag 2147483646 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850688 -20803747 84 [0] PetscCommDuplicate(): returning tag 2147483645 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850688 -20803747 84 [0] PetscCommDuplicate(): returning tag 2147483644 [0] PetscCommDuplicate(): returning tag 2147483643 [0] PetscCommDuplicate(): returning tag 2147483642 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850688 -20803747 84 [0] PetscCommDuplicate(): returning tag 2147483641 atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 [0] PetscCommDuplicate(): returning tag 2147483640 [0] PetscCommDuplicate(): returning tag 2147483639 [0] PetscCommDuplicate(): returning tag 2147483638 [0] PetscCommDuplicate(): returning tag 2147483637 [0] MatSetUpPreallocation(): Warning not preallocating matrix storage [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 24 X 24; storage space: 264 unneeded,5 76 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 48 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 24 [0] Mat_CheckInode(): Found 5 nodes of 24. Limit used: 5. Using Inode routines [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 24 X 24; storage space: 184 unneeded,3 92 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 24 [0] Mat_CheckInode(): Found 15 nodes of 24. Limit used: 5. Using Inode routines [0] PetscCommDuplicate(): returning tag 2147483636 [0] PetscCommDuplicate(): returning tag 2147483635 [0] PetscCommDuplicate(): returning tag 2147483634 [0] PetscCommDuplicate(): returning tag 2147483633 [0] PetscCommDuplicate(): returning tag 2147483632 [0] PCSetUp(): Setting up new PC [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 -2080374783 max tags = 2147483647 [0] PetscCommDuplicate(): returning tag 2147483647 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -20803747 83 [0] PetscCommDuplicate(): returning tag 2147483646 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -20803747 83 [0] PetscCommDuplicate(): returning tag 2147483645 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -20803747 83 [0] PetscCommDuplicate(): returning tag 2147483644 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -20803747 83 [0] PetscCommDuplicate(): returning tag 2147483643 [0] PetscCommDuplicate(): returning tag 2147483631 [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 6.43598e-0 08 is less than relative tolerance 1e-005 times initial right hand side norm 0.0 984377 at iteration 1 [0] SNESSolve_LS(): iter=0, linear solve iterations=1 [0] SNESLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX|| 4.72897e+006 near zero implies inconsistent rhs [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=5.00000 00000000003e-002 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=2.23337 96462448683e-002 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.05579 95926265388e-002 [0] SNESLineSearchCubic(): Cubically determined step, lambda=5.2789979631326939e -003 [0] SNESSolve_LS(): fnorm=7.0710677904340784e+000, gnorm=7.0372413153730617e+000 , ynorm=9.8437692875833122e-002, lssucceed=1 [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 24 X 24; storage space: 56 unneeded,57 6 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 16 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 24 [0] Mat_CheckInode(): Found 5 nodes of 24. Limit used: 5. Using Inode routines [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 24 X 24; storage space: 184 unneeded,3 92 used [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 24 [0] Mat_CheckInode(): Found 15 nodes of 24. Limit used: 5. Using Inode routines [0] PCSetUp(): Setting up PC with same nonzero pattern [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 3.06534e-0 08 is less than relative tolerance 1e-005 times initial right hand side norm 0.0 979013 at iteration 1 [0] SNESSolve_LS(): iter=1, linear solve iterations=1 [0] SNESLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX|| 3.89599e+006 near zero implies inconsistent rhs [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=5.00000 00000000003e-002 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=2.04981 30512175090e-002 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=7.90835 12647314651e-003 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.64025 15420099921e-003 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.64025 15420099923e-004 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=2.40360 10928094253e-005 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=3.57264 27106588782e-006 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=5.33322 51840976446e-007 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=7.91785 19721784329e-008 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.14184 37592725693e-008 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817760e-009 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817761e-010 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817761e-011 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817762e-012 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817764e-013 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817765e-014 [0] SNESLineSearchCubic(): Cubic step no good, shrinking lambda, lambda=1.41828 88220817765e-015 [0] SNESLineSearchCubic(): Unable to find good step length! 18 [0] SNESLineSearchCubic(): fnorm=7.0372413153730617e+000, gnorm=7.03724146468665 79e+000, ynorm=9.7901291669040949e-002, lambda=1.4182888220817765e-015, initial slope=-4.9522772347319197e+001 [0] SNESSolve_LS(): fnorm=7.0372413153730617e+000, gnorm=7.0372414646866579e+000 , ynorm=9.7901291669040949e-002, lssucceed=0 [0] SNESLSCheckLocalMin_Private(): || J^T F|| 3.8948e+006 near zero implies foun d a local minimum STEP 0 (Newton iterations: 2) diverged reason: DIVERGED_LS_FAILURE [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user MPI_Comm 114 0850689 [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374783 [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user MPI_Comm -20 80374783 [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374783 ===============OUTPUT 2================== C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_type test atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 Testing hand-coded Jacobian, if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct. Run with -snes_test_display to show difference of hand-coded and finite difference Jacobian. Norm of matrix ratio 5.91627e-007 difference 23.5238 Norm of matrix ratio 2.1444e-007 difference 8.52641 Norm of matrix ratio 2.49179e-007 difference 9.90765 [0]PETSC ERROR: SNESSolve() line 1871 in src/snes/interface/C:\cygwin\home\PETSC -~1.3-P\src\snes\INTERF~1\snes.c [0]PETSC ERROR: User provided function() line 302 in unknowndirectory/c:\cygwin\ home\rabbit\pgfem - csrbf\domain.cpp [0]PETSC ERROR: User provided function() line 34 in unknowndirectory/c:\cygwin\h ome\rabbit\pgfem - csrbf\main.cpp =================OUTPUT 3=============== C:\cygwin\home\rabbit\pGFEM-test>pgfem -snes_mf_operator -ksp_monitor_true_residual ....... 0 KSP preconditioned resid norm 1.393538348796e-001 true resid norm 7.81673715 5471e-001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 8.503891307640e-002 true resid norm 9.78452660 6728e-001 ||Ae||/||Ax|| 1.251740516806e+000 2 KSP preconditioned resid norm 2.769264336444e-005 true resid norm 3.06983248 7178e+001 ||Ae||/||Ax|| 3.927255613334e+001 3 KSP preconditioned resid norm 2.140259007162e-005 true resid norm 3.58014377 1001e+001 ||Ae||/||Ax|| 4.580099982632e+001 4 KSP preconditioned resid norm 3.271042954922e-006 true resid norm 3.39507468 5413e+000 ||Ae||/||Ax|| 4.343339961273e+000 5 KSP preconditioned resid norm 8.052808588169e-009 true resid norm 5.12770240 1411e-002 ||Ae||/||Ax|| 6.559901272645e-002 29 SNES Function norm 7.688867566940e-001 ........ 0 KSP preconditioned resid norm 9.650772260845e-002 true resid norm 6.46872347 8452e-001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 8.382067825597e-002 true resid norm 8.95465886 5836e-001 ||Ae||/||Ax|| 1.384300765934e+000 2 KSP preconditioned resid norm 2.770083563275e-005 true resid norm 4.75963661 5364e+001 ||Ae||/||Ax|| 7.357922519363e+001 3 KSP preconditioned resid norm 1.746905778563e-005 true resid norm 5.65561314 8780e+001 ||Ae||/||Ax|| 8.743012694265e+001 4 KSP preconditioned resid norm 2.595985260271e-006 true resid norm 4.85638048 7409e+000 ||Ae||/||Ax|| 7.507478876761e+000 5 KSP preconditioned resid norm 1.791711722028e-007 true resid norm 6.13620207 0783e-002 ||Ae||/||Ax|| 9.485955136624e-002 50 SNES Function norm 6.449968664911e-001 STEP 0 (Newton iterations: 50) diverged reason: DIVERGED_MAX_IT C:\cygwin\home\rabbit\pGFEM-test> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 16 07:40:28 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 16 May 2011 14:40:28 +0200 Subject: [petsc-users] SNES convergence issue In-Reply-To: <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at> <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> Message-ID: On Mon, May 16, 2011 at 14:27, Tian(ICT) wrote: > 0 KSP preconditioned resid norm 1.393538348796e-001 true resid norm > 7.81673715 > 5471e-001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 8.503891307640e-002 true resid norm > 9.78452660 > 6728e-001 ||Ae||/||Ax|| 1.251740516806e+000 > 2 KSP preconditioned resid norm 2.769264336444e-005 true resid norm > 3.06983248 > 7178e+001 ||Ae||/||Ax|| 3.927255613334e+001 > 3 KSP preconditioned resid norm 2.140259007162e-005 true resid norm > 3.58014377 > 1001e+001 ||Ae||/||Ax|| 4.580099982632e+001 > 4 KSP preconditioned resid norm 3.271042954922e-006 true resid norm > 3.39507468 > 5413e+000 ||Ae||/||Ax|| 4.343339961273e+000 > 5 KSP preconditioned resid norm 8.052808588169e-009 true resid norm > 5.12770240 > 1411e-002 ||Ae||/||Ax|| 6.559901272645e-002 > 29 SNES Function norm 7.688867566940e-001 The linear systems are not being solved well. Try ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type lu Remove the -pc_type lu if you have a known null space or otherwise expect the system to be singular. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Mon May 16 07:46:06 2011 From: gdiso at ustc.edu (Gong Ding) Date: Mon, 16 May 2011 20:46:06 +0800 (CST) Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> Message-ID: <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> Flexiable AIJ matrix patch file to petsc-3.1-p8. -------------- next part -------------- A non-text attachment was scrubbed... Name: petsc-mat-faij.patch Type: application/octet-stream Size: 122652 bytes Desc: not available URL: From rongtian at ncic.ac.cn Mon May 16 07:54:13 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Mon, 16 May 2011 20:54:13 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> Message-ID: <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> \!\({3.166862696480865`*^7\ Null, 9.748022859341169`*^6\ Null, 6.374098595333358`*^6\ Null, 4.952740471886356`*^6\ Null, 1.6734417698528145`*^6\ Null, 6808.315385343917`\ Null, 2316.3470969822656`\ Null, 1886.5411772849754`\ Null, 1281.6187957520588`\ Null, 1044.3385445169467`\ Null, 133.68138748861358`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 0.6661334228099414`\ Null, 0.26534106700889537`\ Null, 0.0022211797318440266`\ Null, 0.002033972519661367`\ Null, 0.001653668057544512`\ Null}\) The Jacobian is well conditioned, the aboves are the eigenvalues of Jacobian. (ignore Nulls) ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type lu atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067809722e+001 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 7.070972822978e+001 true resid norm 7.070972822978e+001 ||Ae||/||Ax|| 9.999865668458e-001 2 KSP preconditioned resid norm 6.724301992829e+001 true resid norm 6.989750243097e+001 ||Ae||/||Ax|| 9.884999594384e-001 3 KSP preconditioned resid norm 6.716623342588e+001 true resid norm 7.181816951751e+001 ||Ae||/||Ax|| 1.015662293873e+000 4 KSP preconditioned resid norm 5.906940941948e+001 true resid norm 1.016799675661e+002 ||Ae||/||Ax|| 1.437971891972e+000 5 KSP preconditioned resid norm 5.755484385379e+001 true resid norm 2.609971377053e+002 ||Ae||/||Ax|| 3.691056919953e+000 6 KSP preconditioned resid norm 5.755031002601e+001 true resid norm 1.812315583610e+003 ||Ae||/||Ax|| 2.563001278418e+001 7 KSP preconditioned resid norm 5.705010785191e+001 true resid norm 2.339400221451e+003 ||Ae||/||Ax|| 3.308411521998e+001 8 KSP preconditioned resid norm 5.435904473868e+001 true resid norm 4.593026169759e+004 ||Ae||/||Ax|| 6.495519903576e+002 9 KSP preconditioned resid norm 5.369513024767e+001 true resid norm 1.229526591166e+004 ||Ae||/||Ax|| 1.738813181052e+002 10 KSP preconditioned resid norm 5.342258023942e+001 true resid norm 1.105734903438e+005 ||Ae||/||Ax|| 1.563745297305e+003 11 KSP preconditioned resid norm 1.914740074969e+001 true resid norm 2.091222051593e+006 ||Ae||/||Ax|| 2.957434588193e+004 12 KSP preconditioned resid norm 5.794625326821e+000 true resid norm 5.490686672936e+006 ||Ae||/||Ax|| 7.765003561961e+004 13 KSP preconditioned resid norm 7.239823936251e-001 true resid norm 4.262009749173e+005 ||Ae||/||Ax|| 6.027391992073e+003 14 KSP preconditioned resid norm 6.718858761053e-001 true resid norm 7.089916382344e+005 ||Ae||/||Ax|| 1.002665590704e+004 15 KSP preconditioned resid norm 6.284315560280e-001 true resid norm 4.441924320726e+005 ||Ae||/||Ax|| 6.281829619309e+003 16 KSP preconditioned resid norm 6.861179511523e-011 true resid norm 1.032548577293e+006 ||Ae||/||Ax|| 1.460244202260e+004 1 SNES Function norm 6.940037537947e+001 0 KSP preconditioned resid norm 6.940037537947e+001 true resid norm 6.940037537947e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.938917724807e+001 true resid norm 6.938917724807e+001 ||Ae||/||Ax|| 9.998386445126e-001 2 KSP preconditioned resid norm 6.029171691634e+001 true resid norm 1.441086829075e+002 ||Ae||/||Ax|| 2.076482758480e+000 3 KSP preconditioned resid norm 5.766604118606e+001 true resid norm 9.007097199931e+001 ||Ae||/||Ax|| 1.297845602518e+000 4 KSP preconditioned resid norm 5.766567882351e+001 true resid norm 8.577775553964e+001 ||Ae||/||Ax|| 1.235984028482e+000 5 KSP preconditioned resid norm 5.719916885561e+001 true resid norm 8.094529230521e+001 ||Ae||/||Ax|| 1.166352370036e+000 6 KSP preconditioned resid norm 5.696097100530e+001 true resid norm 1.795442636492e+002 ||Ae||/||Ax|| 2.587079142835e+000 7 KSP preconditioned resid norm 5.555754741438e+001 true resid norm 8.478422647294e+002 ||Ae||/||Ax|| 1.221668125127e+001 8 KSP preconditioned resid norm 4.772409261772e+001 true resid norm 5.338467893742e+003 ||Ae||/||Ax|| 7.692275242824e+001 9 KSP preconditioned resid norm 3.571642731108e+001 true resid norm 1.163435398971e+004 ||Ae||/||Ax|| 1.676410815661e+002 10 KSP preconditioned resid norm 2.096423747923e+001 true resid norm 1.059366837544e+006 ||Ae||/||Ax|| 1.526456927288e+004 11 KSP preconditioned resid norm 1.265205200445e+001 true resid norm 1.854575098323e+004 ||Ae||/||Ax|| 2.672283958383e+002 12 KSP preconditioned resid norm 2.067106193291e+000 true resid norm 8.584646820969e+003 ||Ae||/||Ax|| 1.236974119236e+002 13 KSP preconditioned resid norm 2.044680513287e+000 true resid norm 1.394879411501e+004 ||Ae||/||Ax|| 2.009901825278e+002 14 KSP preconditioned resid norm 1.994034645379e+000 true resid norm 1.331028694075e+004 ||Ae||/||Ax|| 1.917898407318e+002 15 KSP preconditioned resid norm 1.704234175501e-001 true resid norm 1.739963344491e+004 ||Ae||/||Ax|| 2.507138232289e+002 16 KSP preconditioned resid norm 4.476463218538e-011 true resid norm 1.887284746671e+004 ||Ae||/||Ax|| 2.719415761588e+002 2 SNES Function norm 6.932995270305e+001 0 KSP preconditioned resid norm 6.932995270305e+001 true resid norm 6.932995270305e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.796589821716e+001 true resid norm 6.796589821716e+001 ||Ae||/||Ax|| 9.803251778964e-001 2 KSP preconditioned resid norm 6.619464654381e+001 true resid norm 7.197343448570e+001 ||Ae||/||Ax|| 1.038129000231e+000 3 KSP preconditioned resid norm 6.448145520233e+001 true resid norm 7.132065758253e+001 ||Ae||/||Ax|| 1.028713489652e+000 4 KSP preconditioned resid norm 6.317324744073e+001 true resid norm 7.754159831869e+001 ||Ae||/||Ax|| 1.118442971551e+000 5 KSP preconditioned resid norm 5.751270629154e+001 true resid norm 2.139311080775e+002 ||Ae||/||Ax|| 3.085695283737e+000 6 KSP preconditioned resid norm 5.727428157396e+001 true resid norm 2.035252565877e+002 ||Ae||/||Ax|| 2.935603568914e+000 7 KSP preconditioned resid norm 5.717357244059e+001 true resid norm 1.452546292007e+003 ||Ae||/||Ax|| 2.095120846581e+001 8 KSP preconditioned resid norm 5.618707993098e+001 true resid norm 1.933732398215e+003 ||Ae||/||Ax|| 2.789173110354e+001 9 KSP preconditioned resid norm 2.577788652214e+001 true resid norm 3.012920881090e+004 ||Ae||/||Ax|| 4.345770858946e+002 10 KSP preconditioned resid norm 2.243616525067e+001 true resid norm 3.445820770263e+004 ||Ae||/||Ax|| 4.970176144533e+002 11 KSP preconditioned resid norm 7.443836886104e+000 true resid norm 2.412374658905e+004 ||Ae||/||Ax|| 3.479556187262e+002 12 KSP preconditioned resid norm 5.709179164603e+000 true resid norm 1.074904042224e+004 ||Ae||/||Ax|| 1.550417965563e+002 13 KSP preconditioned resid norm 5.703921574283e+000 true resid norm 1.301188436241e+004 ||Ae||/||Ax|| 1.876805602067e+002 14 KSP preconditioned resid norm 4.813577737949e+000 true resid norm 6.963974475522e+003 ||Ae||/||Ax|| 1.004468372472e+002 15 KSP preconditioned resid norm 4.723496704426e-002 true resid norm 1.647413677323e+003 ||Ae||/||Ax|| 2.376193280239e+001 16 KSP preconditioned resid norm 6.817500551576e-011 true resid norm 1.281298358008e+003 ||Ae||/||Ax|| 1.848116590381e+001 3 SNES Function norm 6.932995265572e+001 0 KSP preconditioned resid norm 6.932995265572e+001 true resid norm 6.932995265572e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.925610729991e+001 true resid norm 6.925610729991e+001 ||Ae||/||Ax|| 9.989348708173e-001 2 KSP preconditioned resid norm 6.922826578480e+001 true resid norm 6.935709882939e+001 ||Ae||/||Ax|| 1.000391550443e+000 3 KSP preconditioned resid norm 6.634744806213e+001 true resid norm 8.208953917670e+001 ||Ae||/||Ax|| 1.184041471719e+000 4 KSP preconditioned resid norm 5.738421542371e+001 true resid norm 7.414189120750e+001 ||Ae||/||Ax|| 1.069406344119e+000 5 KSP preconditioned resid norm 5.715731293309e+001 true resid norm 7.145438481299e+001 ||Ae||/||Ax|| 1.030642342536e+000 6 KSP preconditioned resid norm 5.671361634587e+001 true resid norm 5.886220999449e+002 ||Ae||/||Ax|| 8.490155804200e+000 7 KSP preconditioned resid norm 5.670545540182e+001 true resid norm 6.293471599551e+002 ||Ae||/||Ax|| 9.077565119369e+000 8 KSP preconditioned resid norm 5.667053161874e+001 true resid norm 3.910919282524e+002 ||Ae||/||Ax|| 5.641024020231e+000 9 KSP preconditioned resid norm 3.488725810207e+001 true resid norm 2.271499085219e+005 ||Ae||/||Ax|| 3.276360358270e+003 10 KSP preconditioned resid norm 3.408598913565e+001 true resid norm 9.437094895339e+005 ||Ae||/||Ax|| 1.361185827171e+004 11 KSP preconditioned resid norm 1.735266234202e+001 true resid norm 2.981335531660e+005 ||Ae||/||Ax|| 4.300212848068e+003 12 KSP preconditioned resid norm 1.439780021726e+001 true resid norm 1.611729374093e+005 ||Ae||/||Ax|| 2.324723027140e+003 13 KSP preconditioned resid norm 1.439483508913e+001 true resid norm 6.414270064883e+004 ||Ae||/||Ax|| 9.251802170895e+002 14 KSP preconditioned resid norm 1.261494970923e+000 true resid norm 3.819893287904e+003 ||Ae||/||Ax|| 5.509730126130e+001 15 KSP preconditioned resid norm 7.986489990995e-003 true resid norm 2.330568872365e+003 ||Ae||/||Ax|| 3.361561321032e+001 16 KSP preconditioned resid norm 4.389616521551e-011 true resid norm 7.946454377036e+003 ||Ae||/||Ax|| 1.146179114891e+002 4 SNES Function norm 6.932995272590e+001 STEP 0 (Newton iterations: 4) diverged reason: DIVERGED_LS_FAILURE -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 16 08:04:14 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 16 May 2011 15:04:14 +0200 Subject: [petsc-users] SNES convergence issue In-Reply-To: <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at> <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> Message-ID: On Mon, May 16, 2011 at 14:54, Tian(ICT) wrote: > 12 KSP preconditioned resid norm 5.794625326821e+000 true resid norm > 5.490686672936e+006 ||Ae||/||Ax|| 7.765003561961e+004 > 13 KSP preconditioned resid norm 7.239823936251e-001 true resid norm > 4.262009749173e+005 ||Ae||/||Ax|| 6.027391992073e+003 > 14 KSP preconditioned resid norm 6.718858761053e-001 true resid norm > 7.089916382344e+005 ||Ae||/||Ax|| 1.002665590704e+004 > 15 KSP preconditioned resid norm 6.284315560280e-001 true resid norm > 4.441924320726e+005 ||Ae||/||Ax|| 6.281829619309e+003 > 16 KSP preconditioned resid norm 6.861179511523e-011 true resid norm > 1.032548577293e+006 ||Ae||/||Ax|| 1.460244202260e+004 > Notice how the true residual is blowing up while the GMRES estimate converges. (The language here is not correct, "preconditioned resid norm" in this case is actually the unpreconditioned norm as estimated by FGMRES.) This indicates that the preconditioner is singular which could be due to lack of pivoting. If you run with -pc_type none, it should converge correctly. Do that and see if SNES is converging properly. Also try -pc_type lu -pc_factor_mat_solver_package superlu (or mumps or umfpack) if you have any of these packages installed. What equations are you solving and with what discretization? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 09:50:13 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Mon, 16 May 2011 22:50:13 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> Message-ID: ./pgfem -pc_type none does not work either, leading to negative jacobian of element. The SNES solve is soling a meshfree method. I attached the segment of code. What is the output for -mat_view, are all the output the jacobian, or do they inlcude preconditioning matrix? I found the output matrices do not have the same nonzero structure. PetscErrorCode FormFunction(SNES snes,Vec u, Vec r, void *ctx) { Log::Send(Log::Info, " Form Function ..."); ApplicationCtx *user = (ApplicationCtx*) ctx; //NOTE: r is the nonlinea function, NOT the r.h.s term //the r.h.s is '-r' //u is the solution during the current Newton-Raphson iteration //not an increment //VecSet(u,0.001); //user->pDomain->pPhyDomain->GetMesh(0)->GetEBoundary()->Treatment(NULL,&u); //printf("solutoin vector.................\n"); //VecView(u,PETSC_VIEWER_STDOUT_WORLD); //dof = du VecAYPX(user->oldu, -1, u); user->pDomain->pPhyDomain->UpdateDofs(user->oldu); VecCopy(u,user->oldu); //x = x + du user->pDomain->pPhyDomain->UpdateConfiguration(user->oldu); //QPoint's gradients in current config, new jacobian, new locations etc. user->pDomain->pPhyDomain->UpdateQPoints(); //mfree connectivity in the new configuration user->pDomain->pPhyDomain->MFreeConnectivity(); //F = (dX/dx)^-1 user->pDomain->pPhyDomain->DeformGradients(); //stress user->pDomain->pPhyDomain->UpdateStress(); //f(x) of J(x) dx = f(x) FormInitialGuess(r); user->pDomain->pPhyDomain->InternalForce(r); VecAssemblyBegin(r); VecAssemblyEnd(r); user->pDomain->pPhyDomain->Loads(user->pDomain->contrl.loadContlType, -1.0*user->lambda, &u, &r); //printf("residual force.................\n"); //VecView(r,PETSC_VIEWER_STDOUT_WORLD); PetscFunctionReturn(0); } PetscErrorCode FormJacobian(SNES snes, Vec x, Mat *jac/*jacobian*/, Mat *B/*precon*/,MatStructure*flag,void *ctx) { Log::Send(Log::Info, " Form Jacobian ..."); ApplicationCtx *user = (ApplicationCtx*) ctx; //initialize jacobian matrix MatZeroEntries(*B); //assemble user->pDomain->pPhyDomain->TangentMatrix(*B); MatAssemblyBegin(*B,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(*B,MAT_FINAL_ASSEMBLY); //boundary treatment, mainly MatZeroRows() user->pDomain->pPhyDomain->Constraints(user->pDomain->contrl.loadContlType, B); //printf("Global matrix........\n"); //MatView(*B,PETSC_VIEWER_STDOUT_WORLD); if (*jac != *B){ MatAssemblyBegin(*jac,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(*jac,MAT_FINAL_ASSEMBLY); } ::DumpMatrixToDisk(*B); *flag = SAME_NONZERO_PATTERN; PetscFunctionReturn(0); } ----- Original Message ----- From: Jed Brown To: PETSc users list Sent: Monday, May 16, 2011 9:04 PM Subject: Re: [petsc-users] SNES convergence issue On Mon, May 16, 2011 at 14:54, Tian(ICT) wrote: 12 KSP preconditioned resid norm 5.794625326821e+000 true resid norm 5.490686672936e+006 ||Ae||/||Ax|| 7.765003561961e+004 13 KSP preconditioned resid norm 7.239823936251e-001 true resid norm 4.262009749173e+005 ||Ae||/||Ax|| 6.027391992073e+003 14 KSP preconditioned resid norm 6.718858761053e-001 true resid norm 7.089916382344e+005 ||Ae||/||Ax|| 1.002665590704e+004 15 KSP preconditioned resid norm 6.284315560280e-001 true resid norm 4.441924320726e+005 ||Ae||/||Ax|| 6.281829619309e+003 16 KSP preconditioned resid norm 6.861179511523e-011 true resid norm 1.032548577293e+006 ||Ae||/||Ax|| 1.460244202260e+004 Notice how the true residual is blowing up while the GMRES estimate converges. (The language here is not correct, "preconditioned resid norm" in this case is actually the unpreconditioned norm as estimated by FGMRES.) This indicates that the preconditioner is singular which could be due to lack of pivoting. If you run with -pc_type none, it should converge correctly. Do that and see if SNES is converging properly. Also try -pc_type lu -pc_factor_mat_solver_package superlu (or mumps or umfpack) if you have any of these packages installed. What equations are you solving and with what discretization? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 16 09:57:58 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 16 May 2011 16:57:58 +0200 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at> <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> Message-ID: On Mon, May 16, 2011 at 16:50, Tian(ICT) wrote: > ./pgfem -pc_type none does not work either, leading to negative jacobian of > element. > During finite differencing or in the line search? Perhaps a trust region would be better for this problem. What is the output for -mat_view, are all the output the jacobian, or do > they inlcude preconditioning matrix? > It includes all matrices. You can call MatSetOptionsPrefix(B,"jacpre_") and then run with -jacpre_mat_view to see only that matrix. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 09:58:15 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Mon, 16 May 2011 22:58:15 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> Message-ID: <0984899E0E0F461987A969CCBE09A32B@rti458laptop> For the SNES solver, the option -mat_view printed out 12 matrices for 4 Newton iterations in a single load step. What these matrices are? I found some are not boundary treated and the rest are. For the formers, they are of course signular. Does this have something to do with the bad convergence issue? Rong -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon May 16 11:21:12 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 11:21:12 -0500 Subject: [petsc-users] SNES convergence issue In-Reply-To: <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> Message-ID: <7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> On May 16, 2011, at 7:54 AM, Tian(ICT) wrote: > \!\({3.166862696480865`*^7\ Null, 9.748022859341169`*^6\ Null, > 6.374098595333358`*^6\ Null, 4.952740471886356`*^6\ Null, > 1.6734417698528145`*^6\ Null, 6808.315385343917`\ Null, > 2316.3470969822656`\ Null, 1886.5411772849754`\ Null, > 1281.6187957520588`\ Null, 1044.3385445169467`\ Null, > 133.68138748861358`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, > 1.`\ Null, 1.`\ Null, 1.`\ Null, 1.`\ Null, 0.6661334228099414`\ Null, > 0.26534106700889537`\ Null, 0.0022211797318440266`\ Null, > 0.002033972519661367`\ Null, 0.001653668057544512`\ Null}\) > The Jacobian is well conditioned, the aboves are the eigenvalues of Jacobian. (ignore Nulls) > > > > ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type lu Please rerun this with -pc_type none > > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067809722e+001 > 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 7.070972822978e+001 true resid norm 7.070972822978e+001 ||Ae||/||Ax|| 9.999865668458e-001 > 2 KSP preconditioned resid norm 6.724301992829e+001 true resid norm 6.989750243097e+001 ||Ae||/||Ax|| 9.884999594384e-001 > 3 KSP preconditioned resid norm 6.716623342588e+001 true resid norm 7.181816951751e+001 ||Ae||/||Ax|| 1.015662293873e+000 > 4 KSP preconditioned resid norm 5.906940941948e+001 true resid norm 1.016799675661e+002 ||Ae||/||Ax|| 1.437971891972e+000 > 5 KSP preconditioned resid norm 5.755484385379e+001 true resid norm 2.609971377053e+002 ||Ae||/||Ax|| 3.691056919953e+000 > 6 KSP preconditioned resid norm 5.755031002601e+001 true resid norm 1.812315583610e+003 ||Ae||/||Ax|| 2.563001278418e+001 > 7 KSP preconditioned resid norm 5.705010785191e+001 true resid norm 2.339400221451e+003 ||Ae||/||Ax|| 3.308411521998e+001 > 8 KSP preconditioned resid norm 5.435904473868e+001 true resid norm 4.593026169759e+004 ||Ae||/||Ax|| 6.495519903576e+002 > 9 KSP preconditioned resid norm 5.369513024767e+001 true resid norm 1.229526591166e+004 ||Ae||/||Ax|| 1.738813181052e+002 > 10 KSP preconditioned resid norm 5.342258023942e+001 true resid norm 1.105734903438e+005 ||Ae||/||Ax|| 1.563745297305e+003 > 11 KSP preconditioned resid norm 1.914740074969e+001 true resid norm 2.091222051593e+006 ||Ae||/||Ax|| 2.957434588193e+004 > 12 KSP preconditioned resid norm 5.794625326821e+000 true resid norm 5.490686672936e+006 ||Ae||/||Ax|| 7.765003561961e+004 > 13 KSP preconditioned resid norm 7.239823936251e-001 true resid norm 4.262009749173e+005 ||Ae||/||Ax|| 6.027391992073e+003 > 14 KSP preconditioned resid norm 6.718858761053e-001 true resid norm 7.089916382344e+005 ||Ae||/||Ax|| 1.002665590704e+004 > 15 KSP preconditioned resid norm 6.284315560280e-001 true resid norm 4.441924320726e+005 ||Ae||/||Ax|| 6.281829619309e+003 > 16 KSP preconditioned resid norm 6.861179511523e-011 true resid norm 1.032548577293e+006 ||Ae||/||Ax|| 1.460244202260e+004 > 1 SNES Function norm 6.940037537947e+001 > 0 KSP preconditioned resid norm 6.940037537947e+001 true resid norm 6.940037537947e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.938917724807e+001 true resid norm 6.938917724807e+001 ||Ae||/||Ax|| 9.998386445126e-001 > 2 KSP preconditioned resid norm 6.029171691634e+001 true resid norm 1.441086829075e+002 ||Ae||/||Ax|| 2.076482758480e+000 > 3 KSP preconditioned resid norm 5.766604118606e+001 true resid norm 9.007097199931e+001 ||Ae||/||Ax|| 1.297845602518e+000 > 4 KSP preconditioned resid norm 5.766567882351e+001 true resid norm 8.577775553964e+001 ||Ae||/||Ax|| 1.235984028482e+000 > 5 KSP preconditioned resid norm 5.719916885561e+001 true resid norm 8.094529230521e+001 ||Ae||/||Ax|| 1.166352370036e+000 > 6 KSP preconditioned resid norm 5.696097100530e+001 true resid norm 1.795442636492e+002 ||Ae||/||Ax|| 2.587079142835e+000 > 7 KSP preconditioned resid norm 5.555754741438e+001 true resid norm 8.478422647294e+002 ||Ae||/||Ax|| 1.221668125127e+001 > 8 KSP preconditioned resid norm 4.772409261772e+001 true resid norm 5.338467893742e+003 ||Ae||/||Ax|| 7.692275242824e+001 > 9 KSP preconditioned resid norm 3.571642731108e+001 true resid norm 1.163435398971e+004 ||Ae||/||Ax|| 1.676410815661e+002 > 10 KSP preconditioned resid norm 2.096423747923e+001 true resid norm 1.059366837544e+006 ||Ae||/||Ax|| 1.526456927288e+004 > 11 KSP preconditioned resid norm 1.265205200445e+001 true resid norm 1.854575098323e+004 ||Ae||/||Ax|| 2.672283958383e+002 > 12 KSP preconditioned resid norm 2.067106193291e+000 true resid norm 8.584646820969e+003 ||Ae||/||Ax|| 1.236974119236e+002 > 13 KSP preconditioned resid norm 2.044680513287e+000 true resid norm 1.394879411501e+004 ||Ae||/||Ax|| 2.009901825278e+002 > 14 KSP preconditioned resid norm 1.994034645379e+000 true resid norm 1.331028694075e+004 ||Ae||/||Ax|| 1.917898407318e+002 > 15 KSP preconditioned resid norm 1.704234175501e-001 true resid norm 1.739963344491e+004 ||Ae||/||Ax|| 2.507138232289e+002 > 16 KSP preconditioned resid norm 4.476463218538e-011 true resid norm 1.887284746671e+004 ||Ae||/||Ax|| 2.719415761588e+002 > 2 SNES Function norm 6.932995270305e+001 > 0 KSP preconditioned resid norm 6.932995270305e+001 true resid norm 6.932995270305e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.796589821716e+001 true resid norm 6.796589821716e+001 ||Ae||/||Ax|| 9.803251778964e-001 > 2 KSP preconditioned resid norm 6.619464654381e+001 true resid norm 7.197343448570e+001 ||Ae||/||Ax|| 1.038129000231e+000 > 3 KSP preconditioned resid norm 6.448145520233e+001 true resid norm 7.132065758253e+001 ||Ae||/||Ax|| 1.028713489652e+000 > 4 KSP preconditioned resid norm 6.317324744073e+001 true resid norm 7.754159831869e+001 ||Ae||/||Ax|| 1.118442971551e+000 > 5 KSP preconditioned resid norm 5.751270629154e+001 true resid norm 2.139311080775e+002 ||Ae||/||Ax|| 3.085695283737e+000 > 6 KSP preconditioned resid norm 5.727428157396e+001 true resid norm 2.035252565877e+002 ||Ae||/||Ax|| 2.935603568914e+000 > 7 KSP preconditioned resid norm 5.717357244059e+001 true resid norm 1.452546292007e+003 ||Ae||/||Ax|| 2.095120846581e+001 > 8 KSP preconditioned resid norm 5.618707993098e+001 true resid norm 1.933732398215e+003 ||Ae||/||Ax|| 2.789173110354e+001 > 9 KSP preconditioned resid norm 2.577788652214e+001 true resid norm 3.012920881090e+004 ||Ae||/||Ax|| 4.345770858946e+002 > 10 KSP preconditioned resid norm 2.243616525067e+001 true resid norm 3.445820770263e+004 ||Ae||/||Ax|| 4.970176144533e+002 > 11 KSP preconditioned resid norm 7.443836886104e+000 true resid norm 2.412374658905e+004 ||Ae||/||Ax|| 3.479556187262e+002 > 12 KSP preconditioned resid norm 5.709179164603e+000 true resid norm 1.074904042224e+004 ||Ae||/||Ax|| 1.550417965563e+002 > 13 KSP preconditioned resid norm 5.703921574283e+000 true resid norm 1.301188436241e+004 ||Ae||/||Ax|| 1.876805602067e+002 > 14 KSP preconditioned resid norm 4.813577737949e+000 true resid norm 6.963974475522e+003 ||Ae||/||Ax|| 1.004468372472e+002 > 15 KSP preconditioned resid norm 4.723496704426e-002 true resid norm 1.647413677323e+003 ||Ae||/||Ax|| 2.376193280239e+001 > 16 KSP preconditioned resid norm 6.817500551576e-011 true resid norm 1.281298358008e+003 ||Ae||/||Ax|| 1.848116590381e+001 > 3 SNES Function norm 6.932995265572e+001 > 0 KSP preconditioned resid norm 6.932995265572e+001 true resid norm 6.932995265572e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.925610729991e+001 true resid norm 6.925610729991e+001 ||Ae||/||Ax|| 9.989348708173e-001 > 2 KSP preconditioned resid norm 6.922826578480e+001 true resid norm 6.935709882939e+001 ||Ae||/||Ax|| 1.000391550443e+000 > 3 KSP preconditioned resid norm 6.634744806213e+001 true resid norm 8.208953917670e+001 ||Ae||/||Ax|| 1.184041471719e+000 > 4 KSP preconditioned resid norm 5.738421542371e+001 true resid norm 7.414189120750e+001 ||Ae||/||Ax|| 1.069406344119e+000 > 5 KSP preconditioned resid norm 5.715731293309e+001 true resid norm 7.145438481299e+001 ||Ae||/||Ax|| 1.030642342536e+000 > 6 KSP preconditioned resid norm 5.671361634587e+001 true resid norm 5.886220999449e+002 ||Ae||/||Ax|| 8.490155804200e+000 > 7 KSP preconditioned resid norm 5.670545540182e+001 true resid norm 6.293471599551e+002 ||Ae||/||Ax|| 9.077565119369e+000 > 8 KSP preconditioned resid norm 5.667053161874e+001 true resid norm 3.910919282524e+002 ||Ae||/||Ax|| 5.641024020231e+000 > 9 KSP preconditioned resid norm 3.488725810207e+001 true resid norm 2.271499085219e+005 ||Ae||/||Ax|| 3.276360358270e+003 > 10 KSP preconditioned resid norm 3.408598913565e+001 true resid norm 9.437094895339e+005 ||Ae||/||Ax|| 1.361185827171e+004 > 11 KSP preconditioned resid norm 1.735266234202e+001 true resid norm 2.981335531660e+005 ||Ae||/||Ax|| 4.300212848068e+003 > 12 KSP preconditioned resid norm 1.439780021726e+001 true resid norm 1.611729374093e+005 ||Ae||/||Ax|| 2.324723027140e+003 > 13 KSP preconditioned resid norm 1.439483508913e+001 true resid norm 6.414270064883e+004 ||Ae||/||Ax|| 9.251802170895e+002 > 14 KSP preconditioned resid norm 1.261494970923e+000 true resid norm 3.819893287904e+003 ||Ae||/||Ax|| 5.509730126130e+001 > 15 KSP preconditioned resid norm 7.986489990995e-003 true resid norm 2.330568872365e+003 ||Ae||/||Ax|| 3.361561321032e+001 > 16 KSP preconditioned resid norm 4.389616521551e-011 true resid norm 7.946454377036e+003 ||Ae||/||Ax|| 1.146179114891e+002 > 4 SNES Function norm 6.932995272590e+001 > STEP 0 (Newton iterations: 4) > > diverged reason: DIVERGED_LS_FAILURE From amesga1 at tigers.lsu.edu Mon May 16 12:39:04 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 16 May 2011 12:39:04 -0500 Subject: [petsc-users] Ksp Null space Message-ID: <854989ED-30EB-42C2-8C6A-C34B39B4323C@tigers.lsu.edu> Dear all, Does anyone know of any example programs that use MatNullSpaceCreate and KSPsetNullSpace routines? Best, Ata -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon May 16 14:02:38 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 14:02:38 -0500 Subject: [petsc-users] Ksp Null space In-Reply-To: <854989ED-30EB-42C2-8C6A-C34B39B4323C@tigers.lsu.edu> References: <854989ED-30EB-42C2-8C6A-C34B39B4323C@tigers.lsu.edu> Message-ID: <49829CFE-3676-47CC-83C6-244AAF2258BF@mcs.anl.gov> src/ksp/ksp/examples/tests/ex20.c Note that this is not a good example from the point of view of efficient generation of the matrix and solution of the linear system but it does demonstrate these functions Barry On May 16, 2011, at 12:39 PM, Ataollah Mesgarnejad wrote: > Dear all, > > Does anyone know of any example programs that use MatNullSpaceCreate and KSPsetNullSpace routines? > > Best, > Ata From bsmith at mcs.anl.gov Mon May 16 14:08:12 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 14:08:12 -0500 Subject: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? In-Reply-To: <88D7E3BB7E1960428303E760100374515F13A729@BL2PRD0103MB055.prod.exchangelabs.com> References: <88D7E3BB7E1960428303E760100374515F13A729@BL2PRD0103MB055.prod.exchangelabs.com> Message-ID: On May 16, 2011, at 1:36 AM, Li, Zhisong (lizs) wrote: > Hi, Petsc Team, > > In my knowledge of Petsc, the PCMG object is the main tool for general multigrid calculations, as DMMG only works for simple constant matrix and RHS. This is not correct. DMMG can be used for non-constant matrix. DMMG is restricted to structured grids only. But if your problem is on a two or three dimensional structured grid you can use it and it is much easier than coding the pieces directly. > But it's difficult to find any example code for using PCMG. I checked out the petsc-user mail archive on this topic but still not find a good description on it. I encounter similar errors in the practice as some previous question-raisers did. > > Here is my simple code implementing the PCMG based on my understanding from the user manual and tutorial. It is supposed to have multigrid of only two levels (finest and coarsest grid). The error message indicates SEGV segmentation violation because of this part of code. Anything wrong with this implementation? > This code looks more or less correct. You need to run with -start_in_debugger to track down what is trigger the SEGV src/ksp/ksp/examples/tutorials/ex19.c is a simple example that runs on two levels. Barry > > DAGetMatrix(da, MATAIJ, &M); > > KSPCreate(PETSC_COMM_WORLD, &ksp); > KSPSetType(ksp, KSPGMRES); > KSPGetPC(ksp, &pcmg); > PCSetType(pcmg, PCMG); > > PCMGSetLevels(pcmg, 2, &PETSC_COMM_WORLD); > PCMGSetType(pcmg, PC_MG_MULTIPLICATIVE); > PCMGSetCycleType(pcmg, PC_MG_CYCLE_W); > PCMGSetCyclesOnLevel(pcmg, 0, 1); > PCMGSetCyclesOnLevel(pcmg, 1, 1); > > PCMGGetCoarseSolve(pcmg, &ksp); > > PCMGGetSmoother(pcmg, 0, &ksp); > PCMGGetSmoother(pcmg, 1, &ksp); > PCMGSetInterpolation(pcmg, 1, M); > PCMGSetRestriction(pcmg, 1, M); > > PCMGSetResidual(pcmg, 0, PCMGDefaultResidual, M); > PCMGSetResidual(pcmg, 1, PCMGDefaultResidual, M); > > > The mandatory remaining PCMGSetRhs( ), PCMGSetX( ) and PCMGSetR( ) functions should be generated by Petsc automatically. > > It is tough to learn some Petsc functions when no detailed example and few guidance is provided. I wish you can add some tutorials codes on PCMG usage in the future version of Petsc. > > > Thank you very much. > > > Zhisong Li > > > From bsmith at mcs.anl.gov Mon May 16 15:40:41 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 15:40:41 -0500 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> Message-ID: We really appreciate the work you've done on this, but 1) we can't accept patches to the released version of PETSc since that is likely outdated compared to petsc-dev. We'd like all patches that can be applied directly to petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html#Sending_patches_to_update_the_ 2) as Jed mentioned in an earlier email. It is sort of over-kill to provide an entire new matrix class with the hash table stuff. If instead it was provided as runtime optional support directly in the SeqAIJ matrices that would be simpler to maintain and for people to use. So you could just add the additional data structure inside the Mat_SeqAIJ and then have MatSetValues_SeqAIJ() use the hash table directly when appropriate. Barry On May 16, 2011, at 7:46 AM, Gong Ding wrote: > Flexiable AIJ matrix patch file to petsc-3.1-p8. > > From qzhu at mcs.anl.gov Mon May 16 15:44:28 2011 From: qzhu at mcs.anl.gov (Qian Zhu) Date: Mon, 16 May 2011 15:44:28 -0500 (CDT) Subject: [petsc-users] running ex29 in snes Message-ID: <1055341461.72073.1305578668442.JavaMail.root@zimbra.anl.gov> Hi, I am trying to run ex29 under $PETSC_DIR/src/snes/examples/tutorials. It seems that it does not terminate even with 80,xxx time steps...I was using the default parameters. How long does this app. to converge? What can I do to make it terminate in a short time? Thanks a lot! Qian -- Qian Zhu, Ph.D. Mathematics and Computer Science Division Argonne National Laboratory 9700 S.Cass Avenue Argonne, IL 60439-4844 Phone: 630-252-0086 From jed at 59A2.org Mon May 16 16:30:18 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 16 May 2011 23:30:18 +0200 Subject: [petsc-users] running ex29 in snes In-Reply-To: <1055341461.72073.1305578668442.JavaMail.root@zimbra.anl.gov> References: <1055341461.72073.1305578668442.JavaMail.root@zimbra.anl.gov> Message-ID: On Mon, May 16, 2011 at 22:44, Qian Zhu wrote: > I am trying to run ex29 under $PETSC_DIR/src/snes/examples/tutorials. It > seems that it does not terminate even with 80,xxx time steps...I was using > the default parameters. How long does this app. to converge? What can I do > to make it terminate in a short time? Thanks a lot! > Maybe try fewer time steps? $ grep max_steps petsc/src/snes/examples/tutorials/ex29.c PetscInt max_steps,print_freq; tsCtx.max_steps = 1000000; tsCtx.print_freq = tsCtx.max_steps; ierr = PetscOptionsGetInt(PETSC_NULL, "-max_st", &tsCtx.max_steps,PETSC_NULL);CHKERRQ(ierr); -------------- next part -------------- An HTML attachment was scrubbed... URL: From amesga1 at tigers.lsu.edu Mon May 16 17:54:15 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 16 May 2011 17:54:15 -0500 Subject: [petsc-users] Ksp Null space In-Reply-To: <49829CFE-3676-47CC-83C6-244AAF2258BF@mcs.anl.gov> References: <854989ED-30EB-42C2-8C6A-C34B39B4323C@tigers.lsu.edu> <49829CFE-3676-47CC-83C6-244AAF2258BF@mcs.anl.gov> Message-ID: Dear Barry, In that example the basis for null space is empty(only constant null space). I was wondering how would I form a Vec for example a for a 2D null space? Specifically I'm trying to remove a null space that its span(C,x,y) in my 2D matrix that I form using DA? Best, Ata On Mon, May 16, 2011 at 2:02 PM, Barry Smith wrote: > > src/ksp/ksp/examples/tests/ex20.c > > Note that this is not a good example from the point of view of efficient > generation of the matrix and solution of the linear system but it does > demonstrate these functions > > Barry > > > On May 16, 2011, at 12:39 PM, Ataollah Mesgarnejad wrote: > > > Dear all, > > > > Does anyone know of any example programs that use MatNullSpaceCreate and > KSPsetNullSpace routines? > > > > Best, > > Ata > > -- A. Mesgarnejad PhD Student, Research Assistant Mechanical Engineering Department Louisiana State University 2203 Patrick F. Taylor Hall Baton Rouge, La 70803 -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 16 17:59:54 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 May 2011 17:59:54 -0500 Subject: [petsc-users] Ksp Null space In-Reply-To: References: <854989ED-30EB-42C2-8C6A-C34B39B4323C@tigers.lsu.edu> <49829CFE-3676-47CC-83C6-244AAF2258BF@mcs.anl.gov> Message-ID: On Mon, May 16, 2011 at 5:54 PM, Ataollah Mesgarnejad < amesga1 at tigers.lsu.edu> wrote: > Dear Barry, > > In that example the basis for null space is empty(only constant null > space). I was wondering how would I form a Vec for example a for a 2D null > space? > Specifically I'm trying to remove a null space that its span(C,x,y) in my > 2D matrix that I form using DA? > Do you mean the null space is translations in x and y? Then you give two vectors, the first with 1s in the first component, and the second with 1s in the second component. These can be easily made using DAVecGetArrayDOF. Matt > Best, > Ata > > On Mon, May 16, 2011 at 2:02 PM, Barry Smith wrote: > >> >> src/ksp/ksp/examples/tests/ex20.c >> >> Note that this is not a good example from the point of view of efficient >> generation of the matrix and solution of the linear system but it does >> demonstrate these functions >> >> Barry >> >> >> On May 16, 2011, at 12:39 PM, Ataollah Mesgarnejad wrote: >> >> > Dear all, >> > >> > Does anyone know of any example programs that use MatNullSpaceCreate and >> KSPsetNullSpace routines? >> > >> > Best, >> > Ata >> >> > > > -- > A. Mesgarnejad > PhD Student, Research Assistant > Mechanical Engineering Department > Louisiana State University > 2203 Patrick F. Taylor Hall > Baton Rouge, La 70803 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 18:08:38 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 17 May 2011 07:08:38 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: <7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> <7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> Message-ID: > ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type > fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol > 1e-10 -pc_type lu >Please rerun this with -pc_type none =====here is the output == ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067809722e+001 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 7.070972822978e+001 true resid norm 7.070972822978e+001 ||Ae||/||Ax|| 9.999865668458e-001 2 KSP preconditioned resid norm 6.724301992829e+001 true resid norm 6.989750223363e+001 ||Ae||/||Ax|| 9.884999566476e-001 3 KSP preconditioned resid norm 6.716623342588e+001 true resid norm 7.181816916652e+001 ||Ae||/||Ax|| 1.015662288909e+000 4 KSP preconditioned resid norm 5.906940941948e+001 true resid norm 1.016799404644e+002 ||Ae||/||Ax|| 1.437971508696e+000 5 KSP preconditioned resid norm 5.755484385379e+001 true resid norm 2.836561864194e+002 ||Ae||/||Ax|| 4.011504260069e+000 6 KSP preconditioned resid norm 5.755031002601e+001 true resid norm 1.812315455897e+003 ||Ae||/||Ax|| 2.563001097805e+001 7 KSP preconditioned resid norm 5.705010785191e+001 true resid norm 2.339399289603e+003 ||Ae||/||Ax|| 3.308410204166e+001 8 KSP preconditioned resid norm 5.435904473868e+001 true resid norm 4.593026052413e+004 ||Ae||/||Ax|| 6.495519737624e+002 9 KSP preconditioned resid norm 5.369513024767e+001 true resid norm 1.229526367172e+004 ||Ae||/||Ax|| 1.738812864277e+002 10 KSP preconditioned resid norm 5.342258023941e+001 true resid norm 1.105734932914e+005 ||Ae||/||Ax|| 1.563745338990e+003 11 KSP preconditioned resid norm 1.914740074967e+001 true resid norm 2.091222043111e+006 ||Ae||/||Ax|| 2.957434576198e+004 12 KSP preconditioned resid norm 5.794625327047e+000 true resid norm 5.490687812505e+006 ||Ae||/||Ax|| 7.765005173555e+004 13 KSP preconditioned resid norm 7.239823935987e-001 true resid norm 4.262008995097e+005 ||Ae||/||Ax|| 6.027390925650e+003 14 KSP preconditioned resid norm 6.718858760910e-001 true resid norm 7.089915894317e+005 ||Ae||/||Ax|| 1.002665521687e+004 15 KSP preconditioned resid norm 6.284315560328e-001 true resid norm 4.441924021925e+005 ||Ae||/||Ax|| 6.281829196742e+003 16 KSP preconditioned resid norm 3.987666568845e-011 true resid norm 1.032548796258e+006 ||Ae||/||Ax|| 1.460244511922e+004 1 SNES Function norm 6.940037526669e+001 0 KSP preconditioned resid norm 6.940037526669e+001 true resid norm 6.940037526669e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.933886242720e+001 true resid norm 6.933886242720e+001 ||Ae||/||Ax|| 9.991136526386e-001 2 KSP preconditioned resid norm 6.891438591388e+001 true resid norm 6.881811457529e+001 ||Ae||/||Ax|| 9.916101218593e-001 3 KSP preconditioned resid norm 6.764095171130e+001 true resid norm 7.066681897777e+001 ||Ae||/||Ax|| 1.018248369785e+000 4 KSP preconditioned resid norm 6.413120239095e+001 true resid norm 7.234812589693e+001 ||Ae||/||Ax|| 1.042474563270e+000 5 KSP preconditioned resid norm 5.725057140055e+001 true resid norm 1.204301773633e+002 ||Ae||/||Ax|| 1.735295766060e+000 6 KSP preconditioned resid norm 5.678550463552e+001 true resid norm 1.005974414617e+002 ||Ae||/||Ax|| 1.449523018789e+000 7 KSP preconditioned resid norm 5.632383529005e+001 true resid norm 1.678515104321e+003 ||Ae||/||Ax|| 2.418596582325e+001 8 KSP preconditioned resid norm 5.534421325099e+001 true resid norm 3.857758870204e+003 ||Ae||/||Ax|| 5.558700302959e+001 9 KSP preconditioned resid norm 5.454380087656e+001 true resid norm 1.728962889866e+004 ||Ae||/||Ax|| 2.491287522902e+002 10 KSP preconditioned resid norm 3.843261545422e+001 true resid norm 4.091913392883e+004 ||Ae||/||Ax|| 5.896096926217e+002 11 KSP preconditioned resid norm 3.842255390405e+001 true resid norm 6.673800140666e+004 ||Ae||/||Ax|| 9.616374717023e+002 12 KSP preconditioned resid norm 1.056444473423e+001 true resid norm 3.645761810658e+003 ||Ae||/||Ax|| 5.253230687368e+001 13 KSP preconditioned resid norm 3.848787976170e+000 true resid norm 7.777734617546e+003 ||Ae||/||Ax|| 1.120704980003e+002 14 KSP preconditioned resid norm 1.816689168952e+000 true resid norm 6.804338709513e+003 ||Ae||/||Ax|| 9.804469620467e+001 15 KSP preconditioned resid norm 1.458923608581e-001 true resid norm 4.822884555081e+004 ||Ae||/||Ax|| 6.949363798895e+002 16 KSP preconditioned resid norm 5.279440363487e-011 true resid norm 1.502765773306e+005 ||Ae||/||Ax|| 2.165356840696e+003 2 SNES Function norm 6.939365111453e+001 0 KSP preconditioned resid norm 6.939365111453e+001 true resid norm 6.939365111453e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.614678302144e+001 true resid norm 6.614678302144e+001 ||Ae||/||Ax|| 9.532108767742e-001 2 KSP preconditioned resid norm 6.216088466859e+001 true resid norm 6.521852076536e+001 ||Ae||/||Ax|| 9.398341162036e-001 3 KSP preconditioned resid norm 5.979869565356e+001 true resid norm 6.747409302444e+001 ||Ae||/||Ax|| 9.723381309492e-001 4 KSP preconditioned resid norm 5.782393132397e+001 true resid norm 6.995476678326e+001 ||Ae||/||Ax|| 1.008085979909e+000 5 KSP preconditioned resid norm 5.774226743240e+001 true resid norm 1.549089153994e+002 ||Ae||/||Ax|| 2.232321154910e+000 6 KSP preconditioned resid norm 5.773986202627e+001 true resid norm 7.721343707679e+001 ||Ae||/||Ax|| 1.112687340076e+000 7 KSP preconditioned resid norm 5.460137391426e+001 true resid norm 2.766614820674e+003 ||Ae||/||Ax|| 3.986841413069e+001 8 KSP preconditioned resid norm 4.747878586716e+001 true resid norm 3.241318880946e+003 ||Ae||/||Ax|| 4.670915608111e+001 9 KSP preconditioned resid norm 3.353025758513e+001 true resid norm 1.004558197159e+004 ||Ae||/||Ax|| 1.447622629772e+002 10 KSP preconditioned resid norm 9.155558584259e+000 true resid norm 7.070924593660e+005 ||Ae||/||Ax|| 1.018958432089e+004 11 KSP preconditioned resid norm 9.142349464189e+000 true resid norm 8.688589822429e+005 ||Ae||/||Ax|| 1.252072730413e+004 12 KSP preconditioned resid norm 8.319531269032e+000 true resid norm 3.681959992624e+005 ||Ae||/||Ax|| 5.305903254099e+003 13 KSP preconditioned resid norm 7.930745170072e+000 true resid norm 8.057961126720e+004 ||Ae||/||Ax|| 1.161195728615e+003 14 KSP preconditioned resid norm 4.617857958566e-001 true resid norm 9.326404675040e+003 ||Ae||/||Ax|| 1.343985296241e+002 15 KSP preconditioned resid norm 1.072818795066e-001 true resid norm 1.783737167715e+005 ||Ae||/||Ax|| 2.570461618702e+003 16 KSP preconditioned resid norm 5.074103600898e-011 true resid norm 1.888074682362e+004 ||Ae||/||Ax|| 2.720817613770e+002 3 SNES Function norm 6.928864174572e+001 0 KSP preconditioned resid norm 6.928864174572e+001 true resid norm 6.928864174572e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.888540894289e+001 true resid norm 6.888540894289e+001 ||Ae||/||Ax|| 9.941803910038e-001 2 KSP preconditioned resid norm 6.663692308768e+001 true resid norm 6.807711217898e+001 ||Ae||/||Ax|| 9.825147450402e-001 3 KSP preconditioned resid norm 6.645029404081e+001 true resid norm 7.451509070572e+001 ||Ae||/||Ax|| 1.075430096886e+000 4 KSP preconditioned resid norm 6.222306576164e+001 true resid norm 7.485798108323e+001 ||Ae||/||Ax|| 1.080378821076e+000 5 KSP preconditioned resid norm 5.750797885167e+001 true resid norm 7.189476666944e+001 ||Ae||/||Ax|| 1.037612584950e+000 6 KSP preconditioned resid norm 5.658321989393e+001 true resid norm 7.994696702224e+002 ||Ae||/||Ax|| 1.153824999422e+001 7 KSP preconditioned resid norm 5.522776509816e+001 true resid norm 2.562056296444e+003 ||Ae||/||Ax|| 3.697656977959e+001 8 KSP preconditioned resid norm 5.506085838020e+001 true resid norm 1.296733696669e+003 ||Ae||/||Ax|| 1.871495333142e+001 9 KSP preconditioned resid norm 4.906593403790e+001 true resid norm 9.850803362130e+004 ||Ae||/||Ax|| 1.421705363814e+003 10 KSP preconditioned resid norm 4.124307116496e+001 true resid norm 1.476858613001e+004 ||Ae||/||Ax|| 2.131458455227e+002 11 KSP preconditioned resid norm 3.571533742713e+001 true resid norm 1.552458633933e+003 ||Ae||/||Ax|| 2.240567277434e+001 12 KSP preconditioned resid norm 2.892118156608e+000 true resid norm 3.116980310277e+004 ||Ae||/||Ax|| 4.498544395942e+002 13 KSP preconditioned resid norm 2.867751455475e+000 true resid norm 2.126012190303e+004 ||Ae||/||Ax|| 3.068341558931e+002 14 KSP preconditioned resid norm 1.177878523029e+000 true resid norm 3.120141661435e+003 ||Ae||/||Ax|| 4.503106978031e+001 15 KSP preconditioned resid norm 7.758975803132e-002 true resid norm 8.312613066242e+003 ||Ae||/||Ax|| 1.199707896822e+002 16 KSP preconditioned resid norm 3.341189821406e-011 true resid norm 1.890871371653e+004 ||Ae||/||Ax|| 2.728977396602e+002 4 SNES Function norm 6.928864174317e+001 0 KSP preconditioned resid norm 6.928864174317e+001 true resid norm 6.928864174317e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.860691783135e+001 true resid norm 6.860691783135e+001 ||Ae||/||Ax|| 9.901611015216e-001 2 KSP preconditioned resid norm 6.713174440610e+001 true resid norm 7.651329483992e+001 ||Ae||/||Ax|| 1.104268938097e+000 3 KSP preconditioned resid norm 6.713081645716e+001 true resid norm 7.361906036098e+001 ||Ae||/||Ax|| 1.062498246594e+000 4 KSP preconditioned resid norm 6.450564344363e+001 true resid norm 1.244950303675e+002 ||Ae||/||Ax|| 1.796759573220e+000 5 KSP preconditioned resid norm 6.212325003952e+001 true resid norm 7.220280195708e+002 ||Ae||/||Ax|| 1.042058267280e+001 6 KSP preconditioned resid norm 5.684851047411e+001 true resid norm 3.616943797102e+002 ||Ae||/||Ax|| 5.220110693624e+000 7 KSP preconditioned resid norm 5.670359259761e+001 true resid norm 2.994671464210e+002 ||Ae||/||Ax|| 4.322023623021e+000 8 KSP preconditioned resid norm 5.550606144364e+001 true resid norm 6.666107538698e+003 ||Ae||/||Ax|| 9.620779641500e+001 9 KSP preconditioned resid norm 4.248981512469e+001 true resid norm 2.225458725300e+004 ||Ae||/||Ax|| 3.211866576269e+002 10 KSP preconditioned resid norm 3.849506980940e+001 true resid norm 4.415045577506e+004 ||Ae||/||Ax|| 6.371961502537e+002 11 KSP preconditioned resid norm 3.599943510712e+001 true resid norm 2.000343955098e+004 ||Ae||/||Ax|| 2.886972387931e+002 12 KSP preconditioned resid norm 1.149775765561e+001 true resid norm 9.235820045177e+003 ||Ae||/||Ax|| 1.332948635277e+002 13 KSP preconditioned resid norm 6.318543781012e+000 true resid norm 2.497153465580e+004 ||Ae||/||Ax|| 3.603986746971e+002 14 KSP preconditioned resid norm 4.981014355778e+000 true resid norm 1.100694829328e+006 ||Ae||/||Ax|| 1.588564592460e+004 15 KSP preconditioned resid norm 1.978453983412e-002 true resid norm 8.666537604794e+003 ||Ae||/||Ax|| 1.250787630809e+002 16 KSP preconditioned resid norm 1.074726601774e-010 true resid norm 1.410990900375e+004 ||Ae||/||Ax|| 2.036395670166e+002 5 SNES Function norm 6.928864178156e+001 STEP 0 (Newton iterations: 5) diverged reason: DIVERGED_LS_FAILURE From jed at 59A2.org Mon May 16 18:20:31 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 17 May 2011 01:20:31 +0200 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at> <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> <7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> Message-ID: On Tue, May 17, 2011 at 01:08, Tian(ICT) wrote: > Please rerun this with -pc_type none >> > > =====here is the output == > > ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres > -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 > I don't see -pc_type none in this command line. Please run: ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 18:25:10 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 17 May 2011 07:25:10 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop><7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> Message-ID: <414BB7970A294FEBAA7339E015E24F35@rti458laptop> ./pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067809722e+001 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.628127802131e+001 true resid norm 6.628127802131e+001 ||Ae||/||Ax|| 9.373588233757e-001 2 KSP preconditioned resid norm 6.060349268020e+001 true resid norm 8.023787055846e+001 ||Ae||/||Ax|| 1.134734847941e+000 3 KSP preconditioned resid norm 5.893436541308e+001 true resid norm 1.661766291099e+002 ||Ae||/||Ax|| 2.350092427079e+000 4 KSP preconditioned resid norm 5.792313231518e+001 true resid norm 6.181515848325e+001 ||Ae||/||Ax|| 8.741983551375e-001 5 KSP preconditioned resid norm 5.765767305092e+001 true resid norm 1.324788033461e+002 ||Ae||/||Ax|| 1.873533204758e+000 6 KSP preconditioned resid norm 5.350115311022e+001 true resid norm 1.072365538549e+003 ||Ae||/||Ax|| 1.516553888898e+001 7 KSP preconditioned resid norm 4.848008978710e+001 true resid norm 2.599831518791e+004 ||Ae||/||Ax|| 3.676716994874e+002 8 KSP preconditioned resid norm 4.791224291681e+001 true resid norm 3.231348947452e+004 ||Ae||/||Ax|| 4.569817507631e+002 9 KSP preconditioned resid norm 4.637212703930e+001 true resid norm 4.373486613183e+004 ||Ae||/||Ax|| 6.185044085094e+002 10 KSP preconditioned resid norm 4.081733203229e+001 true resid norm 2.444631926737e+004 ||Ae||/||Ax|| 3.457231626850e+002 11 KSP preconditioned resid norm 1.014843728188e+000 true resid norm 1.014362613068e+005 ||Ae||/||Ax|| 1.434525365000e+003 12 KSP preconditioned resid norm 4.979596630699e-001 true resid norm 7.361261497711e+005 ||Ae||/||Ax|| 1.041039584939e+004 13 KSP preconditioned resid norm 3.508852039606e-001 true resid norm 5.559326385531e+005 ||Ae||/||Ax|| 7.862074774459e+003 14 KSP preconditioned resid norm 7.777860525083e-002 true resid norm 3.661167090503e+005 ||Ae||/||Ax|| 5.177672155073e+003 15 KSP preconditioned resid norm 5.465765661090e-002 true resid norm 9.610786012254e+005 ||Ae||/||Ax|| 1.359170392771e+004 16 KSP preconditioned resid norm 2.465819452169e-011 true resid norm 2.431750846397e+005 ||Ae||/||Ax|| 3.439015028330e+003 1 SNES Function norm 6.991990621001e+001 0 KSP preconditioned resid norm 6.991990621001e+001 true resid norm 6.991990621001e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.502503132618e+001 true resid norm 6.502503132618e+001 ||Ae||/||Ax|| 9.299931142766e-001 2 KSP preconditioned resid norm 6.053954474707e+001 true resid norm 6.359594931286e+001 ||Ae||/||Ax|| 9.095542708803e-001 3 KSP preconditioned resid norm 5.914372367016e+001 true resid norm 9.831074153838e+001 ||Ae||/||Ax|| 1.406047960692e+000 4 KSP preconditioned resid norm 5.772050909641e+001 true resid norm 8.122529895838e+001 ||Ae||/||Ax|| 1.161690616609e+000 5 KSP preconditioned resid norm 5.765952758116e+001 true resid norm 1.880385124895e+002 ||Ae||/||Ax|| 2.689341600726e+000 6 KSP preconditioned resid norm 5.589748341073e+001 true resid norm 1.817979569657e+003 ||Ae||/||Ax|| 2.600088684611e+001 7 KSP preconditioned resid norm 4.820237841668e+001 true resid norm 6.587294852684e+003 ||Ae||/||Ax|| 9.421200927956e+001 8 KSP preconditioned resid norm 3.172154531581e+001 true resid norm 2.257311945664e+004 ||Ae||/||Ax|| 3.228425305497e+002 9 KSP preconditioned resid norm 1.328602663092e+001 true resid norm 3.591335699194e+004 ||Ae||/||Ax|| 5.136356574059e+002 10 KSP preconditioned resid norm 6.164414352849e+000 true resid norm 4.148412661392e+004 ||Ae||/||Ax|| 5.933092428546e+002 11 KSP preconditioned resid norm 2.376506811812e-001 true resid norm 5.716673227794e+004 ||Ae||/||Ax|| 8.176031029880e+002 12 KSP preconditioned resid norm 1.933722060454e-001 true resid norm 6.354374301149e+004 ||Ae||/||Ax|| 9.088076122503e+002 13 KSP preconditioned resid norm 1.329239756672e-001 true resid norm 1.854229689192e+004 ||Ae||/||Ax|| 2.651933890791e+002 14 KSP preconditioned resid norm 9.100268608279e-002 true resid norm 1.057020985411e+005 ||Ae||/||Ax|| 1.511759730106e+003 15 KSP preconditioned resid norm 6.205834323254e-002 true resid norm 1.610109209270e+005 ||Ae||/||Ax|| 2.302790859635e+003 16 KSP preconditioned resid norm 8.390602143865e-011 true resid norm 2.364535906704e+005 ||Ae||/||Ax|| 3.381777858227e+003 2 SNES Function norm 6.962223710724e+001 0 KSP preconditioned resid norm 6.962223710724e+001 true resid norm 6.962223710724e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.434334113624e+001 true resid norm 6.434334113624e+001 ||Ae||/||Ax|| 9.241780185423e-001 2 KSP preconditioned resid norm 5.941820165437e+001 true resid norm 5.964687589572e+001 ||Ae||/||Ax|| 8.567216219130e-001 3 KSP preconditioned resid norm 5.882629779860e+001 true resid norm 5.838921684707e+001 ||Ae||/||Ax|| 8.386575794330e-001 4 KSP preconditioned resid norm 5.799711404450e+001 true resid norm 6.049268841064e+001 ||Ae||/||Ax|| 8.688702191150e-001 5 KSP preconditioned resid norm 5.774340381874e+001 true resid norm 6.713814119478e+001 ||Ae||/||Ax|| 9.643203663704e-001 6 KSP preconditioned resid norm 5.727007850899e+001 true resid norm 8.752811555840e+002 ||Ae||/||Ax|| 1.257186197904e+001 7 KSP preconditioned resid norm 5.659555300587e+001 true resid norm 5.193886686025e+003 ||Ae||/||Ax|| 7.460097379556e+001 8 KSP preconditioned resid norm 5.493028783520e+001 true resid norm 8.488154761596e+003 ||Ae||/||Ax|| 1.219172941617e+002 9 KSP preconditioned resid norm 3.701021214066e+001 true resid norm 6.281264368395e+004 ||Ae||/||Ax|| 9.021922634748e+002 10 KSP preconditioned resid norm 8.615093281158e+000 true resid norm 1.060694109883e+005 ||Ae||/||Ax|| 1.523499034150e+003 11 KSP preconditioned resid norm 1.998752979796e+000 true resid norm 8.722251359281e+004 ||Ae||/||Ax|| 1.252796767482e+003 12 KSP preconditioned resid norm 7.891593359595e-001 true resid norm 1.293922763518e+005 ||Ae||/||Ax|| 1.858490645058e+003 13 KSP preconditioned resid norm 6.978293692319e-002 true resid norm 4.064264744299e+004 ||Ae||/||Ax|| 5.837595735453e+002 14 KSP preconditioned resid norm 5.897765071088e-002 true resid norm 5.717699035029e+004 ||Ae||/||Ax|| 8.212460950117e+002 15 KSP preconditioned resid norm 3.955451468647e-002 true resid norm 3.877508508813e+004 ||Ae||/||Ax|| 5.569353513936e+002 16 KSP preconditioned resid norm 2.784034002636e-012 true resid norm 6.270335641151e+004 ||Ae||/||Ax|| 9.006225455659e+002 3 SNES Function norm 6.834209913351e+001 0 KSP preconditioned resid norm 6.834209913351e+001 true resid norm 6.834209913351e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.360168969160e+001 true resid norm 6.360168969160e+001 ||Ae||/||Ax|| 9.306370523878e-001 2 KSP preconditioned resid norm 5.903994447863e+001 true resid norm 5.917630489965e+001 ||Ae||/||Ax|| 8.658836302943e-001 3 KSP preconditioned resid norm 5.793852738074e+001 true resid norm 6.434709190569e+001 ||Ae||/||Ax|| 9.415439783315e-001 4 KSP preconditioned resid norm 5.793696280028e+001 true resid norm 6.089964398584e+001 ||Ae||/||Ax|| 8.910999919226e-001 5 KSP preconditioned resid norm 5.781554334063e+001 true resid norm 5.874684577927e+001 ||Ae||/||Ax|| 8.595996687855e-001 6 KSP preconditioned resid norm 5.720593297369e+001 true resid norm 2.104965171999e+003 ||Ae||/||Ax|| 3.080041729310e+001 7 KSP preconditioned resid norm 5.317590421084e+001 true resid norm 9.316738306938e+003 ||Ae||/||Ax|| 1.363250240344e+002 8 KSP preconditioned resid norm 1.381193983226e+001 true resid norm 6.409496773729e+004 ||Ae||/||Ax|| 9.378548295990e+002 9 KSP preconditioned resid norm 2.880689195377e+000 true resid norm 6.988694972227e+004 ||Ae||/||Ax|| 1.022604669865e+003 10 KSP preconditioned resid norm 7.794943146853e-001 true resid norm 7.624373611927e+004 ||Ae||/||Ax|| 1.115618880397e+003 11 KSP preconditioned resid norm 5.678585058560e-001 true resid norm 8.440277048086e+004 ||Ae||/||Ax|| 1.235004068517e+003 12 KSP preconditioned resid norm 5.285416485630e-001 true resid norm 8.073243385373e+004 ||Ae||/||Ax|| 1.181298714516e+003 13 KSP preconditioned resid norm 6.826813027610e-002 true resid norm 8.916609869788e+004 ||Ae||/||Ax|| 1.304702369819e+003 14 KSP preconditioned resid norm 4.395822785796e-002 true resid norm 7.703609683177e+004 ||Ae||/||Ax|| 1.127212915736e+003 15 KSP preconditioned resid norm 4.105844512942e-002 true resid norm 6.293805113425e+004 ||Ae||/||Ax|| 9.209265142896e+002 16 KSP preconditioned resid norm 6.351483010477e-011 true resid norm 5.571352389246e+004 ||Ae||/||Ax|| 8.152152860220e+002 4 SNES Function norm 6.834209911447e+001 0 KSP preconditioned resid norm 6.834209911447e+001 true resid norm 6.834209911447e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.360891652681e+001 true resid norm 6.360891652681e+001 ||Ae||/||Ax|| 9.307427976461e-001 2 KSP preconditioned resid norm 5.946405063671e+001 true resid norm 5.936779903469e+001 ||Ae||/||Ax|| 8.686856242921e-001 3 KSP preconditioned resid norm 5.818882336629e+001 true resid norm 5.929706836849e+001 ||Ae||/||Ax|| 8.676506741352e-001 4 KSP preconditioned resid norm 5.783918311253e+001 true resid norm 5.967679094441e+001 ||Ae||/||Ax|| 8.732068771322e-001 5 KSP preconditioned resid norm 5.782569164472e+001 true resid norm 6.271152573417e+001 ||Ae||/||Ax|| 9.176119338847e-001 6 KSP preconditioned resid norm 5.299269427717e+001 true resid norm 5.191954889542e+003 ||Ae||/||Ax|| 7.597008223065e+001 7 KSP preconditioned resid norm 2.437105872918e+001 true resid norm 1.255636858201e+004 ||Ae||/||Ax|| 1.837281667479e+002 8 KSP preconditioned resid norm 1.583915894206e+001 true resid norm 1.966365014359e+004 ||Ae||/||Ax|| 2.877238246759e+002 9 KSP preconditioned resid norm 1.593243204181e+000 true resid norm 2.410086168344e+004 ||Ae||/||Ax|| 3.526502989478e+002 10 KSP preconditioned resid norm 5.621127413128e-001 true resid norm 2.779449115567e+004 ||Ae||/||Ax|| 4.066964801464e+002 11 KSP preconditioned resid norm 4.230817544937e-001 true resid norm 2.081566084886e+004 ||Ae||/||Ax|| 3.045803555726e+002 12 KSP preconditioned resid norm 1.612848381174e-001 true resid norm 1.258966831462e+004 ||Ae||/||Ax|| 1.842154174038e+002 13 KSP preconditioned resid norm 3.594351162719e-002 true resid norm 9.800765255147e+003 ||Ae||/||Ax|| 1.434074367358e+002 14 KSP preconditioned resid norm 1.988254501052e-002 true resid norm 2.080235797901e+004 ||Ae||/||Ax|| 3.043857043982e+002 15 KSP preconditioned resid norm 4.971087671084e-003 true resid norm 2.123038017870e+004 ||Ae||/||Ax|| 3.106486404981e+002 16 KSP preconditioned resid norm 4.287122275983e-011 true resid norm 1.625148787369e+004 ||Ae||/||Ax|| 2.377961473860e+002 5 SNES Function norm 6.834209907381e+001 0 KSP preconditioned resid norm 6.834209907381e+001 true resid norm 6.834209907381e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.341126471290e+001 true resid norm 6.341126471290e+001 ||Ae||/||Ax|| 9.278507036258e-001 2 KSP preconditioned resid norm 5.967242699409e+001 true resid norm 5.945827321286e+001 ||Ae||/||Ax|| 8.700094673511e-001 3 KSP preconditioned resid norm 5.854991058930e+001 true resid norm 6.196079048968e+001 ||Ae||/||Ax|| 9.066269741402e-001 4 KSP preconditioned resid norm 5.804548354508e+001 true resid norm 8.012882664714e+001 ||Ae||/||Ax|| 1.172466572333e+000 5 KSP preconditioned resid norm 5.791704460777e+001 true resid norm 8.622153286580e+001 ||Ae||/||Ax|| 1.261616690653e+000 6 KSP preconditioned resid norm 5.740687995529e+001 true resid norm 6.578469488664e+002 ||Ae||/||Ax|| 9.625793731560e+000 7 KSP preconditioned resid norm 5.721815782970e+001 true resid norm 1.333898074708e+003 ||Ae||/||Ax|| 1.951795588350e+001 8 KSP preconditioned resid norm 2.731381535031e+001 true resid norm 2.606273150758e+005 ||Ae||/||Ax|| 3.813569068083e+003 9 KSP preconditioned resid norm 1.433089290659e+001 true resid norm 2.189650354673e+005 ||Ae||/||Ax|| 3.203955372087e+003 10 KSP preconditioned resid norm 1.597875604574e+000 true resid norm 8.197762399992e+004 ||Ae||/||Ax|| 1.199518673130e+003 11 KSP preconditioned resid norm 9.021870871274e-001 true resid norm 7.541433994120e+004 ||Ae||/||Ax|| 1.103482933115e+003 12 KSP preconditioned resid norm 3.481527137255e-001 true resid norm 3.919233647992e+004 ||Ae||/||Ax|| 5.734728229168e+002 13 KSP preconditioned resid norm 7.259272713187e-002 true resid norm 3.520333095063e+004 ||Ae||/||Ax|| 5.151046196665e+002 14 KSP preconditioned resid norm 6.997463268374e-002 true resid norm 3.567222539235e+004 ||Ae||/||Ax|| 5.219656094236e+002 15 KSP preconditioned resid norm 6.023016434402e-003 true resid norm 2.939385235018e+004 ||Ae||/||Ax|| 4.300987641371e+002 16 KSP preconditioned resid norm 1.916117348347e-010 true resid norm 2.307520785260e+004 ||Ae||/||Ax|| 3.376426560688e+002 6 SNES Function norm 6.758376343112e+001 0 KSP preconditioned resid norm 6.758376343112e+001 true resid norm 6.758376343112e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.146448787750e+001 true resid norm 6.146448787750e+001 ||Ae||/||Ax|| 9.094564249910e-001 2 KSP preconditioned resid norm 5.967210207836e+001 true resid norm 6.502686309658e+001 ||Ae||/||Ax|| 9.621669435863e-001 3 KSP preconditioned resid norm 5.800538877450e+001 true resid norm 7.518964008270e+001 ||Ae||/||Ax|| 1.112539998743e+000 4 KSP preconditioned resid norm 5.783937338716e+001 true resid norm 9.132731735840e+001 ||Ae||/||Ax|| 1.351320387055e+000 5 KSP preconditioned resid norm 5.783292029189e+001 true resid norm 6.981339953751e+001 ||Ae||/||Ax|| 1.032990706542e+000 6 KSP preconditioned resid norm 5.781968271096e+001 true resid norm 1.892013617231e+002 ||Ae||/||Ax|| 2.799509114581e+000 7 KSP preconditioned resid norm 5.762354654408e+001 true resid norm 1.039408123150e+003 ||Ae||/||Ax|| 1.537955376234e+001 8 KSP preconditioned resid norm 5.177722159096e+001 true resid norm 8.712126639185e+003 ||Ae||/||Ax|| 1.289085750317e+002 9 KSP preconditioned resid norm 3.973147460841e+001 true resid norm 1.219597772435e+004 ||Ae||/||Ax|| 1.804572149461e+002 10 KSP preconditioned resid norm 9.915955997026e+000 true resid norm 1.781266358841e+004 ||Ae||/||Ax|| 2.635642450803e+002 11 KSP preconditioned resid norm 5.431996873451e-001 true resid norm 2.773391125749e+004 ||Ae||/||Ax|| 4.103635229748e+002 12 KSP preconditioned resid norm 4.308747568900e-001 true resid norm 3.321025546288e+004 ||Ae||/||Ax|| 4.913939943094e+002 13 KSP preconditioned resid norm 6.156037081291e-002 true resid norm 3.224962020730e+004 ||Ae||/||Ax|| 4.771799996040e+002 14 KSP preconditioned resid norm 1.464852926852e-002 true resid norm 4.013666893928e+004 ||Ae||/||Ax|| 5.938803479062e+002 15 KSP preconditioned resid norm 4.875572758361e-003 true resid norm 4.795962759108e+004 ||Ae||/||Ax|| 7.096323903294e+002 16 KSP preconditioned resid norm 5.316410910347e-011 true resid norm 4.125546193912e+004 ||Ae||/||Ax|| 6.104345162898e+002 7 SNES Function norm 6.758376335766e+001 0 KSP preconditioned resid norm 6.758376335766e+001 true resid norm 6.758376335766e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.187640076820e+001 true resid norm 6.187640076820e+001 ||Ae||/||Ax|| 9.155512758404e-001 2 KSP preconditioned resid norm 6.111121753615e+001 true resid norm 6.320058228916e+001 ||Ae||/||Ax|| 9.351444659080e-001 3 KSP preconditioned resid norm 6.081916769954e+001 true resid norm 6.255915259371e+001 ||Ae||/||Ax|| 9.256535813585e-001 4 KSP preconditioned resid norm 6.068384108392e+001 true resid norm 3.003679526029e+002 ||Ae||/||Ax|| 4.444380390795e+000 5 KSP preconditioned resid norm 5.922012678126e+001 true resid norm 2.491961952429e+003 ||Ae||/||Ax|| 3.687219871496e+001 6 KSP preconditioned resid norm 5.734180949436e+001 true resid norm 1.096730688440e+003 ||Ae||/||Ax|| 1.622772444080e+001 7 KSP preconditioned resid norm 5.354460609532e+001 true resid norm 2.889051435541e+003 ||Ae||/||Ax|| 4.274771471739e+001 8 KSP preconditioned resid norm 3.767561768216e+001 true resid norm 9.010865464088e+003 ||Ae||/||Ax|| 1.333288502506e+002 9 KSP preconditioned resid norm 3.767270591666e+001 true resid norm 1.042108934512e+004 ||Ae||/||Ax|| 1.541951620831e+002 10 KSP preconditioned resid norm 3.765990892021e+001 true resid norm 1.085324746492e+004 ||Ae||/||Ax|| 1.605895695314e+002 11 KSP preconditioned resid norm 2.079547946116e+001 true resid norm 5.625282386909e+005 ||Ae||/||Ax|| 8.323422827373e+003 12 KSP preconditioned resid norm 9.792242331602e-001 true resid norm 2.100050973239e+005 ||Ae||/||Ax|| 3.107330620412e+003 13 KSP preconditioned resid norm 2.553770094023e-001 true resid norm 1.488463901707e+004 ||Ae||/||Ax|| 2.202398664646e+002 14 KSP preconditioned resid norm 1.748674160610e-001 true resid norm 6.588699561664e+004 ||Ae||/||Ax|| 9.748938553178e+002 15 KSP preconditioned resid norm 5.005909519417e-002 true resid norm 3.602927317840e+004 ||Ae||/||Ax|| 5.331054588915e+002 16 KSP preconditioned resid norm 4.405637361211e-011 true resid norm 3.371003842923e+004 ||Ae||/||Ax|| 4.987890101774e+002 8 SNES Function norm 6.758376330975e+001 0 KSP preconditioned resid norm 6.758376330975e+001 true resid norm 6.758376330975e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.129866549030e+001 true resid norm 6.129866549030e+001 ||Ae||/||Ax|| 9.070028434102e-001 2 KSP preconditioned resid norm 6.005489837610e+001 true resid norm 6.030286975019e+001 ||Ae||/||Ax|| 8.922685982106e-001 3 KSP preconditioned resid norm 5.949054530004e+001 true resid norm 1.453689073848e+002 ||Ae||/||Ax|| 2.150944254444e+000 4 KSP preconditioned resid norm 5.829749609563e+001 true resid norm 3.468802599061e+002 ||Ae||/||Ax|| 5.132597578448e+000 5 KSP preconditioned resid norm 5.783224745902e+001 true resid norm 5.427597354117e+002 ||Ae||/||Ax|| 8.030919097004e+000 6 KSP preconditioned resid norm 5.783216441670e+001 true resid norm 6.488448816479e+002 ||Ae||/||Ax|| 9.600603012799e+000 7 KSP preconditioned resid norm 5.394483397800e+001 true resid norm 6.664533039328e+003 ||Ae||/||Ax|| 9.861145211436e+001 8 KSP preconditioned resid norm 2.744533333744e+001 true resid norm 1.644345548720e+004 ||Ae||/||Ax|| 2.433048217785e+002 9 KSP preconditioned resid norm 6.622016576759e+000 true resid norm 1.962997716209e+004 ||Ae||/||Ax|| 2.904540410412e+002 10 KSP preconditioned resid norm 1.592909314653e+000 true resid norm 2.014541795530e+004 ||Ae||/||Ax|| 2.980807366847e+002 11 KSP preconditioned resid norm 1.070814650392e+000 true resid norm 2.961270093579e+004 ||Ae||/||Ax|| 4.381629475125e+002 12 KSP preconditioned resid norm 1.738510115200e-001 true resid norm 1.929918493783e+004 ||Ae||/||Ax|| 2.855594893314e+002 13 KSP preconditioned resid norm 1.244839135727e-001 true resid norm 2.469644714924e+004 ||Ae||/||Ax|| 3.654198277780e+002 14 KSP preconditioned resid norm 1.196579372862e-001 true resid norm 2.032905532544e+004 ||Ae||/||Ax|| 3.007979184625e+002 15 KSP preconditioned resid norm 5.889294451430e-002 true resid norm 1.927171855070e+004 ||Ae||/||Ax|| 2.851530842160e+002 16 KSP preconditioned resid norm 1.430627309538e-009 true resid norm 2.379091434476e+005 ||Ae||/||Ax|| 3.520211538934e+003 9 SNES Function norm 6.758376348032e+001 STEP 0 (Newton iterations: 9) diverged reason: DIVERGED_LS_FAILURE -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 16 18:35:57 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 17 May 2011 01:35:57 +0200 Subject: [petsc-users] SNES convergence issue In-Reply-To: <414BB7970A294FEBAA7339E015E24F35@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at> <96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop> <0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop> <7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov> <414BB7970A294FEBAA7339E015E24F35@rti458laptop> Message-ID: On Tue, May 17, 2011 at 01:25, Tian(ICT) wrote: > pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres > -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type > none > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067809722e+001 > 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm > 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.628127802131e+001 true resid norm > 6.628127802131e+001 ||Ae||/||Ax|| 9.373588233757e-001 > 2 KSP preconditioned resid norm 6.060349268020e+001 true resid norm > 8.023787055846e+001 ||Ae||/||Ax|| 1.134734847941e+000 > 3 KSP preconditioned resid norm 5.893436541308e+001 true resid norm > 1.661766291099e+002 ||Ae||/||Ax|| 2.350092427079e+000 > This is remarkably early loss of orthogonality. Does it do the same thing if you remove -snes_mf_operator? ./pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none If so, add -ksp_view_binary and send the matrix (probably named "binaryoutput") to petsc-maint at mcs.anl.gov . ./pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none -ksp_view_binary -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Mon May 16 20:04:42 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 17 May 2011 09:04:42 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop><7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov><414BB7970A294FEBAA7339E015E24F35@rti458laptop> Message-ID: <8628D7CE4DBD4365B7DEF73F4E01404F@rti458laptop> RE: Does it do the same thing if you remove -snes_mf_operator? the output is attached: ./pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067809722e+001 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.487377579675e+001 true resid norm 6.487377579675e+001 ||Ae||/||Ax|| 9.174537360193e-001 2 KSP preconditioned resid norm 5.946077820276e+001 true resid norm 5.946077820276e+001 ||Ae||/||Ax|| 8.409023898909e-001 3 KSP preconditioned resid norm 5.819067642395e+001 true resid norm 5.819067642395e+001 ||Ae||/||Ax|| 8.229404382736e-001 4 KSP preconditioned resid norm 5.782978385330e+001 true resid norm 5.782978385330e+001 ||Ae||/||Ax|| 8.178366465923e-001 5 KSP preconditioned resid norm 5.781901277269e+001 true resid norm 5.781901277269e+001 ||Ae||/||Ax|| 8.176843205094e-001 6 KSP preconditioned resid norm 5.206124924795e+001 true resid norm 5.206124924794e+001 ||Ae||/||Ax|| 7.362572478284e-001 7 KSP preconditioned resid norm 4.698325359049e+001 true resid norm 4.698325359046e+001 ||Ae||/||Ax|| 6.644435445218e-001 8 KSP preconditioned resid norm 2.252503506952e+001 true resid norm 2.252503506964e+001 ||Ae||/||Ax|| 3.185521009807e-001 9 KSP preconditioned resid norm 6.934718778605e+000 true resid norm 6.934718778631e+000 ||Ae||/||Ax|| 9.807173350955e-002 10 KSP preconditioned resid norm 2.116841820236e-001 true resid norm 2.116841820007e-001 ||Ae||/||Ax|| 2.993666412160e-003 11 KSP preconditioned resid norm 2.248667031844e-003 true resid norm 2.248666418030e-003 ||Ae||/||Ax|| 3.180094546595e-005 12 KSP preconditioned resid norm 1.982189998703e-003 true resid norm 1.982189392148e-003 ||Ae||/||Ax|| 2.803239122418e-005 13 KSP preconditioned resid norm 7.277701878363e-005 true resid norm 7.277706020826e-005 ||Ae||/||Ax|| 1.029223056074e-006 14 KSP preconditioned resid norm 5.491342233241e-005 true resid norm 5.491344683351e-005 ||Ae||/||Ax|| 7.765934129213e-007 15 KSP preconditioned resid norm 1.985911495745e-006 true resid norm 1.987317308127e-006 ||Ae||/||Ax|| 2.810491090744e-008 16 KSP preconditioned resid norm 8.806822122196e-010 true resid norm 2.040010932359e-009 ||Ae||/||Ax|| 2.885011128806e-011 1 SNES Function norm 7.037540247543e+001 0 KSP preconditioned resid norm 7.037540247543e+001 true resid norm 7.037540247543e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.014879337697e+001 true resid norm 6.014879337697e+001 ||Ae||/||Ax|| 8.546848936029e-001 2 KSP preconditioned resid norm 5.936532946785e+001 true resid norm 5.936532946785e+001 ||Ae||/||Ax|| 8.435522551872e-001 3 KSP preconditioned resid norm 5.806472337970e+001 true resid norm 5.806472337970e+001 ||Ae||/||Ax|| 8.250712796986e-001 4 KSP preconditioned resid norm 5.776376194749e+001 true resid norm 5.776376194749e+001 ||Ae||/||Ax|| 8.207947651547e-001 5 KSP preconditioned resid norm 5.775321493826e+001 true resid norm 5.775321493826e+001 ||Ae||/||Ax|| 8.206448973194e-001 6 KSP preconditioned resid norm 5.200122358332e+001 true resid norm 5.200122358332e+001 ||Ae||/||Ax|| 7.389119174342e-001 7 KSP preconditioned resid norm 4.692806395721e+001 true resid norm 4.692806395717e+001 ||Ae||/||Ax|| 6.668248039300e-001 8 KSP preconditioned resid norm 2.249722526208e+001 true resid norm 2.249722526213e+001 ||Ae||/||Ax|| 3.196745520566e-001 9 KSP preconditioned resid norm 6.926914352987e+000 true resid norm 6.926914353057e+000 ||Ae||/||Ax|| 9.842806022282e-002 10 KSP preconditioned resid norm 2.112669106382e-001 true resid norm 2.112669106262e-001 ||Ae||/||Ax|| 3.001999323555e-003 11 KSP preconditioned resid norm 2.205894521142e-003 true resid norm 2.205894348027e-003 ||Ae||/||Ax|| 3.134467826024e-005 12 KSP preconditioned resid norm 1.997274432688e-003 true resid norm 1.997274203505e-003 ||Ae||/||Ax|| 2.838028818666e-005 13 KSP preconditioned resid norm 1.074423963192e-003 true resid norm 1.074423723438e-003 ||Ae||/||Ax|| 1.526703486794e-005 14 KSP preconditioned resid norm 2.196883599631e-008 true resid norm 2.193274946006e-008 ||Ae||/||Ax|| 3.116536273838e-010 15 KSP preconditioned resid norm 6.701870711306e-010 true resid norm 2.507972672018e-009 ||Ae||/||Ax|| 3.563706328918e-011 2 SNES Function norm 7.036650604408e+001 0 KSP preconditioned resid norm 7.036650604408e+001 true resid norm 7.036650604408e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 5.961660984718e+001 true resid norm 5.961660984718e+001 ||Ae||/||Ax|| 8.472299279693e-001 2 KSP preconditioned resid norm 5.898186321330e+001 true resid norm 5.898186321330e+001 ||Ae||/||Ax|| 8.382093488676e-001 3 KSP preconditioned resid norm 5.803844203063e+001 true resid norm 5.803844203063e+001 ||Ae||/||Ax|| 8.248021010773e-001 4 KSP preconditioned resid norm 5.775247508524e+001 true resid norm 5.775247508524e+001 ||Ae||/||Ax|| 8.207381371055e-001 5 KSP preconditioned resid norm 5.774197168106e+001 true resid norm 5.774197168106e+001 ||Ae||/||Ax|| 8.205888700071e-001 6 KSP preconditioned resid norm 5.199097115954e+001 true resid norm 5.199097115953e+001 ||Ae||/||Ax|| 7.388596376656e-001 7 KSP preconditioned resid norm 4.691863775585e+001 true resid norm 4.691863775585e+001 ||Ae||/||Ax|| 6.667751518948e-001 8 KSP preconditioned resid norm 2.249247574480e+001 true resid norm 2.249247574465e+001 ||Ae||/||Ax|| 3.196474716331e-001 9 KSP preconditioned resid norm 6.925581352168e+000 true resid norm 6.925581352175e+000 ||Ae||/||Ax|| 9.842156078967e-002 10 KSP preconditioned resid norm 2.111959897559e-001 true resid norm 2.111959897926e-001 ||Ae||/||Ax|| 3.001370988355e-003 11 KSP preconditioned resid norm 2.197554118182e-003 true resid norm 2.197554117525e-003 ||Ae||/||Ax|| 3.123011559148e-005 12 KSP preconditioned resid norm 1.997607369649e-003 true resid norm 1.997607414609e-003 ||Ae||/||Ax|| 2.838861166927e-005 13 KSP preconditioned resid norm 1.000748904690e-003 true resid norm 1.000748843027e-003 ||Ae||/||Ax|| 1.422194875499e-005 14 KSP preconditioned resid norm 8.525193893681e-007 true resid norm 8.526861878498e-007 ||Ae||/||Ax|| 1.211778494893e-008 15 KSP preconditioned resid norm 5.457483213140e-009 true resid norm 7.175955157596e-009 ||Ae||/||Ax|| 1.019796997324e-010 3 SNES Function norm 7.036650590325e+001 0 KSP preconditioned resid norm 7.036650590325e+001 true resid norm 7.036650590325e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 5.961656312518e+001 true resid norm 5.961656312518e+001 ||Ae||/||Ax|| 8.472292656843e-001 2 KSP preconditioned resid norm 5.898128385953e+001 true resid norm 5.898128385953e+001 ||Ae||/||Ax|| 8.382011171710e-001 3 KSP preconditioned resid norm 5.803842974968e+001 true resid norm 5.803842974968e+001 ||Ae||/||Ax|| 8.248019281997e-001 4 KSP preconditioned resid norm 5.775247225502e+001 true resid norm 5.775247225502e+001 ||Ae||/||Ax|| 8.207380985270e-001 5 KSP preconditioned resid norm 5.774196886175e+001 true resid norm 5.774196886175e+001 ||Ae||/||Ax|| 8.205888315833e-001 6 KSP preconditioned resid norm 5.199096858866e+001 true resid norm 5.199096858865e+001 ||Ae||/||Ax|| 7.388596026089e-001 7 KSP preconditioned resid norm 4.691863539211e+001 true resid norm 4.691863539209e+001 ||Ae||/||Ax|| 6.667751196372e-001 8 KSP preconditioned resid norm 2.249247455418e+001 true resid norm 2.249247455410e+001 ||Ae||/||Ax|| 3.196474553536e-001 9 KSP preconditioned resid norm 6.925581017184e+000 true resid norm 6.925581017176e+000 ||Ae||/||Ax|| 9.842155622589e-002 10 KSP preconditioned resid norm 2.111959721825e-001 true resid norm 2.111959721155e-001 ||Ae||/||Ax|| 3.001370743147e-003 11 KSP preconditioned resid norm 2.197552999267e-003 true resid norm 2.197553329081e-003 ||Ae||/||Ax|| 3.123010444916e-005 12 KSP preconditioned resid norm 1.997607044290e-003 true resid norm 1.997607318404e-003 ||Ae||/||Ax|| 2.838861035889e-005 13 KSP preconditioned resid norm 1.000461310492e-003 true resid norm 1.000461552464e-003 ||Ae||/||Ax|| 1.421786600915e-005 14 KSP preconditioned resid norm 8.488175184617e-007 true resid norm 8.484518586716e-007 ||Ae||/||Ax|| 1.205760962237e-008 15 KSP preconditioned resid norm 5.589522141624e-009 true resid norm 5.386565191049e-009 ||Ae||/||Ax|| 7.655013023463e-011 4 SNES Function norm 7.036650608027e+001 STEP 0 (Newton iterations: 4) diverged reason: DIVERGED_LS_FAILURE -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon May 16 20:37:49 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 20:37:49 -0500 Subject: [petsc-users] SNES convergence issue In-Reply-To: <8628D7CE4DBD4365B7DEF73F4E01404F@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop><7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov><414BB7970A294FEBAA7339E015E24F35@rti458laptop> <8628D7CE4DBD4365B7DEF73F4E01404F@rti458laptop> Message-ID: My conclusion based on the two sets of output (with -pc_type none and -snes_mf_operator and then without the -snes_mf_operator) I need to conclude that 1) the function is bad (likely a bug in the code)** and 2) the Jacobian is possibly wrong, that is it is not a Jacobian for the function. Reason: The finite differencing of the function (with -snes_mf_operator) gives as Jed says "This is remarkably early loss of orthogonality." Now if the differencing doesn't work this is usually because the function code is wrong. Now because the explicit Jacobian linear solve actually works but Newton's method doesn't converge that could indicate the Jacobian is wrong. But since the function is suspicious it could just be do to the bad function. ** of course sometimes finite differencing doesn't work for the correctly coded function but usually that is because the function is hugely nonlinear which is not common for most PDEs people solve. Suggest you work on your function evaluation to make sure it makes sense (for example run with valgrind: http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind) Barry From Jed's email. On Tue, May 17, 2011 at 01:25, Tian(ICT) wrote: pgfem -snes_monitor -snes_mf_operator -mat_mffd_type ds -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067809722e+001 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.628127802131e+001 true resid norm 6.628127802131e+001 ||Ae||/||Ax|| 9.373588233757e-001 2 KSP preconditioned resid norm 6.060349268020e+001 true resid norm 8.023787055846e+001 ||Ae||/||Ax|| 1.134734847941e+000 3 KSP preconditioned resid norm 5.893436541308e+001 true resid norm 1.661766291099e+002 ||Ae||/||Ax|| 2.350092427079e+000 This is remarkably early loss of orthogonality. Does it do the same thing if you remove -snes_mf_operator? ./pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none If so, add -ksp_view_binary and send the matrix (probably named "binaryoutput") to petsc-maint at mcs.anl.gov . On May 16, 2011, at 8:04 PM, Tian(ICT) wrote: > RE: Does it do the same thing if you remove -snes_mf_operator? > the output is attached: > > ./pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none > > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067809722e+001 > 0 KSP preconditioned resid norm 7.071067809722e+001 true resid norm 7.071067809722e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.487377579675e+001 true resid norm 6.487377579675e+001 ||Ae||/||Ax|| 9.174537360193e-001 > 2 KSP preconditioned resid norm 5.946077820276e+001 true resid norm 5.946077820276e+001 ||Ae||/||Ax|| 8.409023898909e-001 > 3 KSP preconditioned resid norm 5.819067642395e+001 true resid norm 5.819067642395e+001 ||Ae||/||Ax|| 8.229404382736e-001 > 4 KSP preconditioned resid norm 5.782978385330e+001 true resid norm 5.782978385330e+001 ||Ae||/||Ax|| 8.178366465923e-001 > 5 KSP preconditioned resid norm 5.781901277269e+001 true resid norm 5.781901277269e+001 ||Ae||/||Ax|| 8.176843205094e-001 > 6 KSP preconditioned resid norm 5.206124924795e+001 true resid norm 5.206124924794e+001 ||Ae||/||Ax|| 7.362572478284e-001 > 7 KSP preconditioned resid norm 4.698325359049e+001 true resid norm 4.698325359046e+001 ||Ae||/||Ax|| 6.644435445218e-001 > 8 KSP preconditioned resid norm 2.252503506952e+001 true resid norm 2.252503506964e+001 ||Ae||/||Ax|| 3.185521009807e-001 > 9 KSP preconditioned resid norm 6.934718778605e+000 true resid norm 6.934718778631e+000 ||Ae||/||Ax|| 9.807173350955e-002 > 10 KSP preconditioned resid norm 2.116841820236e-001 true resid norm 2.116841820007e-001 ||Ae||/||Ax|| 2.993666412160e-003 > 11 KSP preconditioned resid norm 2.248667031844e-003 true resid norm 2.248666418030e-003 ||Ae||/||Ax|| 3.180094546595e-005 > 12 KSP preconditioned resid norm 1.982189998703e-003 true resid norm 1.982189392148e-003 ||Ae||/||Ax|| 2.803239122418e-005 > 13 KSP preconditioned resid norm 7.277701878363e-005 true resid norm 7.277706020826e-005 ||Ae||/||Ax|| 1.029223056074e-006 > 14 KSP preconditioned resid norm 5.491342233241e-005 true resid norm 5.491344683351e-005 ||Ae||/||Ax|| 7.765934129213e-007 > 15 KSP preconditioned resid norm 1.985911495745e-006 true resid norm 1.987317308127e-006 ||Ae||/||Ax|| 2.810491090744e-008 > 16 KSP preconditioned resid norm 8.806822122196e-010 true resid norm 2.040010932359e-009 ||Ae||/||Ax|| 2.885011128806e-011 > 1 SNES Function norm 7.037540247543e+001 > 0 KSP preconditioned resid norm 7.037540247543e+001 true resid norm 7.037540247543e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.014879337697e+001 true resid norm 6.014879337697e+001 ||Ae||/||Ax|| 8.546848936029e-001 > 2 KSP preconditioned resid norm 5.936532946785e+001 true resid norm 5.936532946785e+001 ||Ae||/||Ax|| 8.435522551872e-001 > 3 KSP preconditioned resid norm 5.806472337970e+001 true resid norm 5.806472337970e+001 ||Ae||/||Ax|| 8.250712796986e-001 > 4 KSP preconditioned resid norm 5.776376194749e+001 true resid norm 5.776376194749e+001 ||Ae||/||Ax|| 8.207947651547e-001 > 5 KSP preconditioned resid norm 5.775321493826e+001 true resid norm 5.775321493826e+001 ||Ae||/||Ax|| 8.206448973194e-001 > 6 KSP preconditioned resid norm 5.200122358332e+001 true resid norm 5.200122358332e+001 ||Ae||/||Ax|| 7.389119174342e-001 > 7 KSP preconditioned resid norm 4.692806395721e+001 true resid norm 4.692806395717e+001 ||Ae||/||Ax|| 6.668248039300e-001 > 8 KSP preconditioned resid norm 2.249722526208e+001 true resid norm 2.249722526213e+001 ||Ae||/||Ax|| 3.196745520566e-001 > 9 KSP preconditioned resid norm 6.926914352987e+000 true resid norm 6.926914353057e+000 ||Ae||/||Ax|| 9.842806022282e-002 > 10 KSP preconditioned resid norm 2.112669106382e-001 true resid norm 2.112669106262e-001 ||Ae||/||Ax|| 3.001999323555e-003 > 11 KSP preconditioned resid norm 2.205894521142e-003 true resid norm 2.205894348027e-003 ||Ae||/||Ax|| 3.134467826024e-005 > 12 KSP preconditioned resid norm 1.997274432688e-003 true resid norm 1.997274203505e-003 ||Ae||/||Ax|| 2.838028818666e-005 > 13 KSP preconditioned resid norm 1.074423963192e-003 true resid norm 1.074423723438e-003 ||Ae||/||Ax|| 1.526703486794e-005 > 14 KSP preconditioned resid norm 2.196883599631e-008 true resid norm 2.193274946006e-008 ||Ae||/||Ax|| 3.116536273838e-010 > 15 KSP preconditioned resid norm 6.701870711306e-010 true resid norm 2.507972672018e-009 ||Ae||/||Ax|| 3.563706328918e-011 > 2 SNES Function norm 7.036650604408e+001 > 0 KSP preconditioned resid norm 7.036650604408e+001 true resid norm 7.036650604408e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 5.961660984718e+001 true resid norm 5.961660984718e+001 ||Ae||/||Ax|| 8.472299279693e-001 > 2 KSP preconditioned resid norm 5.898186321330e+001 true resid norm 5.898186321330e+001 ||Ae||/||Ax|| 8.382093488676e-001 > 3 KSP preconditioned resid norm 5.803844203063e+001 true resid norm 5.803844203063e+001 ||Ae||/||Ax|| 8.248021010773e-001 > 4 KSP preconditioned resid norm 5.775247508524e+001 true resid norm 5.775247508524e+001 ||Ae||/||Ax|| 8.207381371055e-001 > 5 KSP preconditioned resid norm 5.774197168106e+001 true resid norm 5.774197168106e+001 ||Ae||/||Ax|| 8.205888700071e-001 > 6 KSP preconditioned resid norm 5.199097115954e+001 true resid norm 5.199097115953e+001 ||Ae||/||Ax|| 7.388596376656e-001 > 7 KSP preconditioned resid norm 4.691863775585e+001 true resid norm 4.691863775585e+001 ||Ae||/||Ax|| 6.667751518948e-001 > 8 KSP preconditioned resid norm 2.249247574480e+001 true resid norm 2.249247574465e+001 ||Ae||/||Ax|| 3.196474716331e-001 > 9 KSP preconditioned resid norm 6.925581352168e+000 true resid norm 6.925581352175e+000 ||Ae||/||Ax|| 9.842156078967e-002 > 10 KSP preconditioned resid norm 2.111959897559e-001 true resid norm 2.111959897926e-001 ||Ae||/||Ax|| 3.001370988355e-003 > 11 KSP preconditioned resid norm 2.197554118182e-003 true resid norm 2.197554117525e-003 ||Ae||/||Ax|| 3.123011559148e-005 > 12 KSP preconditioned resid norm 1.997607369649e-003 true resid norm 1.997607414609e-003 ||Ae||/||Ax|| 2.838861166927e-005 > 13 KSP preconditioned resid norm 1.000748904690e-003 true resid norm 1.000748843027e-003 ||Ae||/||Ax|| 1.422194875499e-005 > 14 KSP preconditioned resid norm 8.525193893681e-007 true resid norm 8.526861878498e-007 ||Ae||/||Ax|| 1.211778494893e-008 > 15 KSP preconditioned resid norm 5.457483213140e-009 true resid norm 7.175955157596e-009 ||Ae||/||Ax|| 1.019796997324e-010 > 3 SNES Function norm 7.036650590325e+001 > 0 KSP preconditioned resid norm 7.036650590325e+001 true resid norm 7.036650590325e+001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 5.961656312518e+001 true resid norm 5.961656312518e+001 ||Ae||/||Ax|| 8.472292656843e-001 > 2 KSP preconditioned resid norm 5.898128385953e+001 true resid norm 5.898128385953e+001 ||Ae||/||Ax|| 8.382011171710e-001 > 3 KSP preconditioned resid norm 5.803842974968e+001 true resid norm 5.803842974968e+001 ||Ae||/||Ax|| 8.248019281997e-001 > 4 KSP preconditioned resid norm 5.775247225502e+001 true resid norm 5.775247225502e+001 ||Ae||/||Ax|| 8.207380985270e-001 > 5 KSP preconditioned resid norm 5.774196886175e+001 true resid norm 5.774196886175e+001 ||Ae||/||Ax|| 8.205888315833e-001 > 6 KSP preconditioned resid norm 5.199096858866e+001 true resid norm 5.199096858865e+001 ||Ae||/||Ax|| 7.388596026089e-001 > 7 KSP preconditioned resid norm 4.691863539211e+001 true resid norm 4.691863539209e+001 ||Ae||/||Ax|| 6.667751196372e-001 > 8 KSP preconditioned resid norm 2.249247455418e+001 true resid norm 2.249247455410e+001 ||Ae||/||Ax|| 3.196474553536e-001 > 9 KSP preconditioned resid norm 6.925581017184e+000 true resid norm 6.925581017176e+000 ||Ae||/||Ax|| 9.842155622589e-002 > 10 KSP preconditioned resid norm 2.111959721825e-001 true resid norm 2.111959721155e-001 ||Ae||/||Ax|| 3.001370743147e-003 > 11 KSP preconditioned resid norm 2.197552999267e-003 true resid norm 2.197553329081e-003 ||Ae||/||Ax|| 3.123010444916e-005 > 12 KSP preconditioned resid norm 1.997607044290e-003 true resid norm 1.997607318404e-003 ||Ae||/||Ax|| 2.838861035889e-005 > 13 KSP preconditioned resid norm 1.000461310492e-003 true resid norm 1.000461552464e-003 ||Ae||/||Ax|| 1.421786600915e-005 > 14 KSP preconditioned resid norm 8.488175184617e-007 true resid norm 8.484518586716e-007 ||Ae||/||Ax|| 1.205760962237e-008 > 15 KSP preconditioned resid norm 5.589522141624e-009 true resid norm 5.386565191049e-009 ||Ae||/||Ax|| 7.655013023463e-011 > 4 SNES Function norm 7.036650608027e+001 > STEP 0 (Newton iterations: 4) > > diverged reason: DIVERGED_LS_FAILURE From gdiso at ustc.edu Mon May 16 21:16:19 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 17 May 2011 10:16:19 +0800 (CST) Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> Message-ID: <14631630.27291305598579219.JavaMail.coremail@mail.ustc.edu> > We really appreciate the work you've done on this, but > > > > 1) we can't accept patches to the released version of PETSc since that is likely outdated compared to petsc-dev. We'd like all patches that can be applied directly to petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html#Sending_patches_to_update_the_ I did the edit based on petsc-3.1. OK, I will shift to petsc-dev. Our company always use Git, so I need to spend some time on Mercurial. > 2) as Jed mentioned in an earlier email. It is sort of over-kill to provide an entire new matrix class with the hash table stuff. If instead it was provided as runtime optional support directly in the SeqAIJ matrices that would be simpler to maintain and for people to use. So you could just add the additional data structure inside the Mat_SeqAIJ and then have MatSetValues_SeqAIJ() use the hash table directly when appropriate. That's great if hash table can be introduced in AIJ. I used to think that your guys reject any changes to basic petsc objects. From bsmith at mcs.anl.gov Mon May 16 21:22:41 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 21:22:41 -0500 Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: <14631630.27291305598579219.JavaMail.coremail@mail.ustc.edu> References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> <14631630.27291305598579219.JavaMail.coremail@mail.ustc.edu> Message-ID: On May 16, 2011, at 9:16 PM, Gong Ding wrote: >> We really appreciate the work you've done on this, but >> >> >> >> 1) we can't accept patches to the released version of PETSc since that is likely outdated compared to petsc-dev. We'd like all patches that can be applied directly to petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html#Sending_patches_to_update_the_ > > I did the edit based on petsc-3.1. OK, I will shift to petsc-dev. > Our company always use Git, so I need to spend some time on Mercurial. > > >> 2) as Jed mentioned in an earlier email. It is sort of over-kill to provide an entire new matrix class with the hash table stuff. If instead it was provided as runtime optional support directly in the SeqAIJ matrices that would be simpler to maintain and for people to use. So you could just add the additional data structure inside the Mat_SeqAIJ and then have MatSetValues_SeqAIJ() use the hash table directly when appropriate. > > That's great if hash table can be introduced in AIJ. > I used to think that your guys reject any changes to basic petsc objects. In this case I think it is better to put in the basic object rather than introduce an entire new class since everyone will benefit from it immediately in the basic class. Thanks barry > > From sylbar.vainbot at gmail.com Mon May 16 21:52:22 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Mon, 16 May 2011 19:52:22 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> Message-ID: Hi Barry, Thanks for your comments. >> ? ? ? CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); > > ? ?You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. My function matrixsmoothfv indeed performs the smoothing operation so it not a multiplication. I can see no documentation about MATOP_SOR, is there an example somewhere online? Assigning MATOP_SOR to my smoothing matrix now generates the following runtime error message: [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in src/mat/impls/shell/ftn-custom/zshellf.c application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 rank 0 in job 37 catalina.gps.caltech.edu_50093 caused collective abort of all ranks exit status of rank 0: return code 1 >> I also provide work space... >> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >> ERROR: PCSetUp_BJacobi()". > > It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. ?What solver options are you running with? I setup the solver with: CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); but this error might not be representative of the problem. upstream, my smoothing matrices were not set up properly. >> MatGetVecs() line 7265 in src/mat/interface/matrix.c >> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >> [0]PETSC ERROR: KSPSetUp() line >> > ? ? This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. From knepley at gmail.com Mon May 16 21:54:17 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 May 2011 21:54:17 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: On Fri, May 13, 2011 at 10:16 AM, Barry Smith wrote: > > On May 13, 2011, at 9:34 AM, Jed Brown wrote: > > > How small is the Reynolds number? There is a difference between 0.1 and > 100, although both may be laminar. > > > > On Fri, May 13, 2011 at 15:50, Henning Sauerland > wrote: > > The problem is discretized using FEM (more precisely XFEM) with > stabilized, trilinear hexahedral elements. As the XFEM approximation space > is time-dependant as well as the physical properties at the nodes the > resulting system may change quite significantly between time steps. > > > > I assume the XFEM basis is just resolving the jump across the interface. > Does this mean the size of the matrix is changing as the interface moves? In > any case, it looks like you can't amortize setup costs between time steps, > so we need a solution short of a direct solve. > > > > ILU(2) requires less than half the number of KSP iterations, but it > scales similar to ILU(0) and requires about 1/3 more time. > > > I think this is a more important point then you and Jed may give it > credit for. It is not clear to me that the worse convergence (of the linear > solve) with the number of subdomains is the chief issue you should be > concerned about. An equally important issue is that the ILU(0) is not a good > preconditioner (and while user a more direct solver, LU or ILU(2)) helps the > convergence it is too expensive). Are you using AIJ matrix (and hence it is > point ILU?) see below Anecdotally, I hear that XFEM can produce ill-conditioned matrices, where the conditioning arises from geometric factors. I don't think you can a priori rule out differences in conditioning as the process count increases which come from changing geometry of the interface. Matt > > > > Eventually, we're going to need a coarse level to battle the poor scaling > with more processes. Hopefully that will alleviate the need for these more > expensive local solves. > > > > I guess you are talking about the nonlinear iterations? I was always > referring to the KSP iterations and I thought that the ksp iteration count > grows with increasing number of processors is more or less solely related to > the iterative solver and preconditioner. > > > > I meant linear iterations. It is mostly dependent on preconditioner. The > increased iteration count (when a direct subdomain solver is used, inexact > subdomain solves confuse things) is likely due to the fundamental scaling > for elliptic problems. There are many ways to improve constants, but you > need a coarse level to fix the asymptotics. > > > > Unfortunately, multigrid methods for XFEM are a recent topic. Perhaps the > best results I have seen (at conferences) use some geometric information in > an otherwise algebraic framework. For this problem (unlike many fracture > problems), the influence of the extended basis functions may be local enough > that you can build a coarse level using the conventional part of the > problem. The first thing I would try is probably to see if a direct solve > with the conventional part makes an effective coarse level. If that works, I > would see if ML or Hypre can do a reasonable job with that part of the > problem. > > > > I have no great confidence that this will work, it's highly dependent on > how local the influence of the extended basis functions is. Perhaps you have > enough experience with the method to hypothesize. > > > > Note: for the conventional part of the problem, it is still > incompressible flow. It sounds like you are using equal-order elements > (Q1-Q1 stabilized; PSPG or Dohrmann&Bochev?). For those elements, you will > want to interlace the velocity and pressure degrees of freedom, then use a > smoother that respects the block size. > > Is Jed is correct on this? That you have equal order elements hence all > the degrees of freedom live at the same points, then you can switch to BAIJ > matrix format and ILU becomes automatically point block ILU and it may work > much better as Jed indicates. But you can make this change before mucking > with coarse grid solves or multilevel methods. I'm guess that the > point-block ILU will be a good bit better and since this is easily checked > (just create a BAIJ matrix and set the block size appropriately and leave > the rest of the code unchanged (well interlace the variables if they are not > currently interlaced). > > Barry > > > > PETSc's ML and Hypre interfaces forward this information if you set the > block size on the matrix. When using ML, you'll probably have to make the > smoothers stronger. There are also some "energy minimization" options that > may help. > > ibcgs is slightly faster, requiring less number of ksp iterations > compared to lgmres. Unfortunately, the iteration count scales very similar > to lgmres and generally the lack of robustness of bcgs solvers turns out to > problematic for tougher testcases in my experience. > > > > Yes, that suggestion was only an attempt to improve the constants a > little. > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon May 16 23:33:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 16 May 2011 23:33:00 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> Message-ID: <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> On May 16, 2011, at 9:52 PM, Sylvain Barbot wrote: > Hi Barry, > > Thanks for your comments. > >>> CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); >> >> You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. > > My function matrixsmoothfv indeed performs the smoothing operation so > it not a multiplication. I can see no documentation about MATOP_SOR, > is there an example somewhere online? For each possible Matrix function there is the name, for matrix vector multiply it is MATOP_MULT and for smoothing it is MATOP_SOR and the calling sequence you use in your function that does the operation is the same as the matrix operation. For example for MatMult() it is Mat,Vec,Vec,PetscErrorCode (see below for the smoother). > Assigning MATOP_SOR to my > smoothing matrix now generates the following runtime error message: > > [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in > src/mat/impls/shell/ftn-custom/zshellf.c > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 > rank 0 in job 37 catalina.gps.caltech.edu_50093 caused collective > abort of all ranks > exit status of rank 0: return code 1 This is because we hadn't added support for providing the MATOP_SOR for the Fortran interface. Edit src/mat/imples/shell/ftn-custom/zshellf.c and locate the function matshellsetoperation_() and put instead of it the following code. static PetscErrorCode oursor(Mat mat,Vec b,PetscReal omega,MatSORType flg,PetscReal shift,PetscInt its,PetscInt lits,Vec x) { PetscErrorCode ierr = 0; (*(PetscErrorCode (PETSC_STDCALL *)(Mat*,Vec*,PetscReal*,MatSORType*,PetscReal*,PetscInt*,PetscInt*,Vec*,PetscErrorCode*))(((PetscObject)mat)->fortran_func_pointers[9]))(&mat,&b,&omega,&flg,&shift,&its,&lits,&x,&ierr); return ierr; } void PETSC_STDCALL matshellsetoperation_(Mat *mat,MatOperation *op,PetscErrorCode (PETSC_STDCALL *f)(Mat*,Vec*,Vec*,PetscErrorCode*),PetscErrorCode *ierr) { MPI_Comm comm; *ierr = PetscObjectGetComm((PetscObject)*mat,&comm);if (*ierr) return; PetscObjectAllocateFortranPointers(*mat,10); if (*op == MATOP_MULT) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmult); ((PetscObject)*mat)->fortran_func_pointers[0] = (PetscVoidFunction)f; } else if (*op == MATOP_MULT_TRANSPOSE) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttranspose); ((PetscObject)*mat)->fortran_func_pointers[2] = (PetscVoidFunction)f; } else if (*op == MATOP_MULT_ADD) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmultadd); ((PetscObject)*mat)->fortran_func_pointers[1] = (PetscVoidFunction)f; } else if (*op == MATOP_MULT_TRANSPOSE_ADD) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttransposeadd); ((PetscObject)*mat)->fortran_func_pointers[3] = (PetscVoidFunction)f; } else if (*op == MATOP_GET_DIAGONAL) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetdiagonal); ((PetscObject)*mat)->fortran_func_pointers[4] = (PetscVoidFunction)f; } else if (*op == MATOP_DIAGONAL_SCALE) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalscale); ((PetscObject)*mat)->fortran_func_pointers[5] = (PetscVoidFunction)f; } else if (*op == MATOP_DIAGONAL_SET) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalset); ((PetscObject)*mat)->fortran_func_pointers[6] = (PetscVoidFunction)f; } else if (*op == MATOP_GET_VECS) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetvecs); ((PetscObject)*mat)->fortran_func_pointers[7] = (PetscVoidFunction)f; } else if (*op == MATOP_VIEW) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourview); ((PetscObject)*mat)->fortran_func_pointers[8] = (PetscVoidFunction)f; } else if (*op == MATOP_SOR) { *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)oursor); ((PetscObject)*mat)->fortran_func_pointers[9] = (PetscVoidFunction)f; } else { PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL, "Cannot set that matrix operation"); *ierr = 1; } } Then in that directory run make to update the PETSc libraries. Now note the calling sequence of the relaxation function you need to provide. It is PetscErrorCode MatSOR(Mat mat,Vec b,PetscReal omega,MatSORType flag,PetscReal shift,PetscInt its,PetscInt lits,Vec x,PetscErrorCode ierr) in your function you can ignore the omega, the flag, and the shift argument. You function should do its*lits smoothing steps. Once you have this going please let us know what happens. Barry > >>> I also provide work space... >>> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >>> ERROR: PCSetUp_BJacobi()". >> >> It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. What solver options are you running with? > > I setup the solver with: > > CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); > > but this error might not be representative of the problem. upstream, > my smoothing matrices were not set up properly. > >>> MatGetVecs() line 7265 in src/mat/interface/matrix.c >>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>> [0]PETSC ERROR: KSPSetUp() line >>> >> This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. From rongtian at ncic.ac.cn Tue May 17 01:09:09 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Tue, 17 May 2011 14:09:09 +0800 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop><7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov><414BB7970A294FEBAA7339E015E24F35@rti458laptop> Message-ID: I am using SNES to solve a superelastic large deformatoin beam problem. The model contains only one brick element. I am solving the problem using both a FE and a RBF meshfree approximation. The code flows in the same way and switching between the two different approximation is just a change in shape function. The FE method converges well and matched with another code's results. However, the rbf meshfree method does not converge. I combed the code and could not find a reason. The following is the output. Anyone can give me a hint what is happening. c:\pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 0 SNES Function norm 7.071067597552e-001 0 KSP preconditioned resid norm 7.071067597552e-001 true resid norm 7.071067597552e-001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.487378720919e-001 true resid norm 6.487378720919e-001 ||Ae||/||Ax|| 9.174539249441e-001 2 KSP preconditioned resid norm 5.946077657343e-001 true resid norm 5.946077657343e-001 ||Ae||/||Ax|| 8.409023920802e-001 3 KSP preconditioned resid norm 5.819067797177e-001 true resid norm 5.819067797177e-001 ||Ae||/||Ax|| 8.229404848557e-001 4 KSP preconditioned resid norm 5.782978365298e-001 true resid norm 5.782978365298e-001 ||Ae||/||Ax|| 8.178366682988e-001 5 KSP preconditioned resid norm 5.781901258865e-001 true resid norm 5.781901258865e-001 ||Ae||/||Ax|| 8.176843424417e-001 6 KSP preconditioned resid norm 5.206124902964e-001 true resid norm 5.206124902965e-001 ||Ae||/||Ax|| 7.362572668330e-001 7 KSP preconditioned resid norm 4.698325347773e-001 true resid norm 4.698325347774e-001 ||Ae||/||Ax|| 6.644435628646e-001 8 KSP preconditioned resid norm 2.252503531363e-001 true resid norm 2.252503531382e-001 ||Ae||/||Ax|| 3.185521139923e-001 9 KSP preconditioned resid norm 6.934719137364e-002 true resid norm 6.934719137322e-002 ||Ae||/||Ax|| 9.807174152489e-002 10 KSP preconditioned resid norm 2.116841871461e-003 true resid norm 2.116841872136e-003 ||Ae||/||Ax|| 2.993666575708e-003 11 KSP preconditioned resid norm 2.248666383380e-005 true resid norm 2.248666456589e-005 ||Ae||/||Ax|| 3.180094696545e-005 12 KSP preconditioned resid norm 1.982194861438e-005 true resid norm 1.982195167981e-005 ||Ae||/||Ax|| 2.803247374791e-005 13 KSP preconditioned resid norm 7.277668728205e-007 true resid norm 7.277725200859e-007 ||Ae||/||Ax|| 1.029225799422e-006 14 KSP preconditioned resid norm 5.491308415661e-007 true resid norm 5.491254698377e-007 ||Ae||/||Ax|| 7.765807104259e-007 15 KSP preconditioned resid norm 1.986082449735e-008 true resid norm 1.986464500270e-008 ||Ae||/||Ax|| 2.809285122600e-008 16 KSP preconditioned resid norm 2.868546981451e-012 true resid norm 2.473921585844e-011 ||Ae||/||Ax|| 3.498653564987e-011 1 SNES Function norm 6.961586744390e-001 0 KSP preconditioned resid norm 6.961586744390e-001 true resid norm 6.961586744390e-001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 5.912814321436e-001 true resid norm 5.912814321436e-001 ||Ae||/||Ax|| 8.493486526188e-001 2 KSP preconditioned resid norm 5.867863600876e-001 true resid norm 5.867863600876e-001 ||Ae||/||Ax|| 8.428916878189e-001 3 KSP preconditioned resid norm 5.743057651077e-001 true resid norm 5.743057651077e-001 ||Ae||/||Ax|| 8.249638856696e-001 4 KSP preconditioned resid norm 5.713990222419e-001 true resid norm 5.713990222419e-001 ||Ae||/||Ax|| 8.207884828877e-001 5 KSP preconditioned resid norm 5.712949806941e-001 true resid norm 5.712949806941e-001 ||Ae||/||Ax|| 8.206390319772e-001 6 KSP preconditioned resid norm 5.144041331498e-001 true resid norm 5.144041331499e-001 ||Ae||/||Ax|| 7.389179393110e-001 7 KSP preconditioned resid norm 4.642286939843e-001 true resid norm 4.642286939846e-001 ||Ae||/||Ax|| 6.668432227160e-001 8 KSP preconditioned resid norm 2.225622386414e-001 true resid norm 2.225622386428e-001 ||Ae||/||Ax|| 3.197004459108e-001 9 KSP preconditioned resid norm 6.852028979252e-002 true resid norm 6.852028979329e-002 ||Ae||/||Ax|| 9.842625296383e-002 10 KSP preconditioned resid norm 2.091417649890e-003 true resid norm 2.091417651046e-003 ||Ae||/||Ax|| 3.004225513287e-003 11 KSP preconditioned resid norm 2.221359037659e-005 true resid norm 2.221359037215e-005 ||Ae||/||Ax|| 3.190880353542e-005 12 KSP preconditioned resid norm 1.804209918914e-005 true resid norm 1.804209941359e-005 ||Ae||/||Ax|| 2.591664813792e-005 13 KSP preconditioned resid norm 1.533714545172e-005 true resid norm 1.533714905817e-005 ||Ae||/||Ax|| 2.203111104021e-005 14 KSP preconditioned resid norm 5.645319291054e-008 true resid norm 5.645404908731e-008 ||Ae||/||Ax|| 8.109365172072e-008 15 KSP preconditioned resid norm 3.193476220728e-010 true resid norm 3.097685921426e-010 ||Ae||/||Ax|| 4.449683721779e-010 16 KSP preconditioned resid norm 3.809088792508e-012 true resid norm 2.536256355736e-011 ||Ae||/||Ax|| 3.643215905885e-011 2 SNES Function norm 6.943721478945e-001 0 KSP preconditioned resid norm 6.943721478945e-001 true resid norm 6.943721478945e-001 ||Ae||/||Ax|| 1.000000000000e+000 1 KSP preconditioned resid norm 6.312241812914e-001 true resid norm 6.312241812914e-001 ||Ae||/||Ax|| 9.090574603338e-001 2 KSP preconditioned resid norm 5.826977509711e-001 true resid norm 5.826977509711e-001 ||Ae||/||Ax|| 8.391721251177e-001 3 KSP preconditioned resid norm 5.690491242974e-001 true resid norm 5.690491242974e-001 ||Ae||/||Ax|| 8.195160563725e-001 4 KSP preconditioned resid norm 5.668884553347e-001 true resid norm 5.668884553347e-001 ||Ae||/||Ax|| 8.164043691178e-001 5 KSP preconditioned resid norm 5.667877314429e-001 true resid norm 5.667877314429e-001 ||Ae||/||Ax|| 8.162593116120e-001 6 KSP preconditioned resid norm 5.103464760431e-001 true resid norm 5.103464760434e-001 ||Ae||/||Ax|| 7.349754416143e-001 7 KSP preconditioned resid norm 4.605661569558e-001 true resid norm 4.605661569565e-001 ||Ae||/||Ax|| 6.632843185791e-001 8 KSP preconditioned resid norm 2.208053445289e-001 true resid norm 2.208053445308e-001 ||Ae||/||Ax|| 3.179928014111e-001 9 KSP preconditioned resid norm 6.797979448157e-002 true resid norm 6.797979448302e-002 ||Ae||/||Ax|| 9.790109624810e-002 10 KSP preconditioned resid norm 2.074808319295e-003 true resid norm 2.074808318610e-003 ||Ae||/||Ax|| 2.988035054259e-003 11 KSP preconditioned resid norm 2.203325006674e-005 true resid norm 2.203324811647e-005 ||Ae||/||Ax|| 3.173118072677e-005 12 KSP preconditioned resid norm 1.823511305062e-005 true resid norm 1.823511045569e-005 ||Ae||/||Ax|| 2.626129304147e-005 13 KSP preconditioned resid norm 1.735630734709e-005 true resid norm 1.735630720647e-005 ||Ae||/||Ax|| 2.499568460385e-005 14 KSP preconditioned resid norm 2.741404670294e-008 true resid norm 2.741193558629e-008 ||Ae||/||Ax|| 3.947729710849e-008 15 KSP preconditioned resid norm 1.660262277254e-010 true resid norm 1.728810158968e-010 ||Ae||/||Ax|| 2.489745828962e-010 16 KSP preconditioned resid norm 4.907918572393e-012 true resid norm 1.668357561843e-011 ||Ae||/||Ax|| 2.402685025461e-011 3 SNES Function norm 6.943722078847e-001 STEP 0 (Newton iterations: 3) diverged reason: DIVERGED_LS_FAILURE -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue May 17 08:13:44 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 17 May 2011 08:13:44 -0500 Subject: [petsc-users] SNES convergence issue In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><96F735AC4EAF4736BC190A0C8C643B1B@rti458laptop><0D2451AFA800428EB59B1F05A0AEB84C@rti458laptop><7A4D501C-7A47-4043-AB03-365483F69900@mcs.anl.gov><414BB7970A294FEBAA7339E015E24F35@rti458laptop> Message-ID: <17F5B854-36DE-426D-9296-833EE0EAF45E@mcs.anl.gov> Gregory E. Fasshauer has a book and set of Matlab routines for meshfree methods http://www.mathworks.com/support/books/book48894.html?category=new Perhaps you can use those to mimimc the rbf meshfree method you have coded and determine what has gone wrong. Barry On May 17, 2011, at 1:09 AM, Tian(ICT) wrote: > I am using SNES to solve a superelastic large deformatoin beam problem. The model contains only one brick element. > I am solving the problem using both a FE and a RBF meshfree approximation. The code flows in the same way and switching between > the two different approximation is just a change in shape function. The FE method converges well and matched with another code's results. > However, the rbf meshfree method does not converge. I combed the code and could not find a reason. > The following is the output. Anyone can give me a hint what is happening. > > c:\pgfem -snes_monitor -ksp_type fgmres -ksp_monitor_true_residual -ksp_gmres_modifiedgramschmidt -ksp_rtol 1e-10 -pc_type none > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067597552e-001 > 0 KSP preconditioned resid norm 7.071067597552e-001 true resid norm 7.071067597552e-001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.487378720919e-001 true resid norm 6.487378720919e-001 ||Ae||/||Ax|| 9.174539249441e-001 > 2 KSP preconditioned resid norm 5.946077657343e-001 true resid norm 5.946077657343e-001 ||Ae||/||Ax|| 8.409023920802e-001 > 3 KSP preconditioned resid norm 5.819067797177e-001 true resid norm 5.819067797177e-001 ||Ae||/||Ax|| 8.229404848557e-001 > 4 KSP preconditioned resid norm 5.782978365298e-001 true resid norm 5.782978365298e-001 ||Ae||/||Ax|| 8.178366682988e-001 > 5 KSP preconditioned resid norm 5.781901258865e-001 true resid norm 5.781901258865e-001 ||Ae||/||Ax|| 8.176843424417e-001 > 6 KSP preconditioned resid norm 5.206124902964e-001 true resid norm 5.206124902965e-001 ||Ae||/||Ax|| 7.362572668330e-001 > 7 KSP preconditioned resid norm 4.698325347773e-001 true resid norm 4.698325347774e-001 ||Ae||/||Ax|| 6.644435628646e-001 > 8 KSP preconditioned resid norm 2.252503531363e-001 true resid norm 2.252503531382e-001 ||Ae||/||Ax|| 3.185521139923e-001 > 9 KSP preconditioned resid norm 6.934719137364e-002 true resid norm 6.934719137322e-002 ||Ae||/||Ax|| 9.807174152489e-002 > 10 KSP preconditioned resid norm 2.116841871461e-003 true resid norm 2.116841872136e-003 ||Ae||/||Ax|| 2.993666575708e-003 > 11 KSP preconditioned resid norm 2.248666383380e-005 true resid norm 2.248666456589e-005 ||Ae||/||Ax|| 3.180094696545e-005 > 12 KSP preconditioned resid norm 1.982194861438e-005 true resid norm 1.982195167981e-005 ||Ae||/||Ax|| 2.803247374791e-005 > 13 KSP preconditioned resid norm 7.277668728205e-007 true resid norm 7.277725200859e-007 ||Ae||/||Ax|| 1.029225799422e-006 > 14 KSP preconditioned resid norm 5.491308415661e-007 true resid norm 5.491254698377e-007 ||Ae||/||Ax|| 7.765807104259e-007 > 15 KSP preconditioned resid norm 1.986082449735e-008 true resid norm 1.986464500270e-008 ||Ae||/||Ax|| 2.809285122600e-008 > 16 KSP preconditioned resid norm 2.868546981451e-012 true resid norm 2.473921585844e-011 ||Ae||/||Ax|| 3.498653564987e-011 > 1 SNES Function norm 6.961586744390e-001 > 0 KSP preconditioned resid norm 6.961586744390e-001 true resid norm 6.961586744390e-001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 5.912814321436e-001 true resid norm 5.912814321436e-001 ||Ae||/||Ax|| 8.493486526188e-001 > 2 KSP preconditioned resid norm 5.867863600876e-001 true resid norm 5.867863600876e-001 ||Ae||/||Ax|| 8.428916878189e-001 > 3 KSP preconditioned resid norm 5.743057651077e-001 true resid norm 5.743057651077e-001 ||Ae||/||Ax|| 8.249638856696e-001 > 4 KSP preconditioned resid norm 5.713990222419e-001 true resid norm 5.713990222419e-001 ||Ae||/||Ax|| 8.207884828877e-001 > 5 KSP preconditioned resid norm 5.712949806941e-001 true resid norm 5.712949806941e-001 ||Ae||/||Ax|| 8.206390319772e-001 > 6 KSP preconditioned resid norm 5.144041331498e-001 true resid norm 5.144041331499e-001 ||Ae||/||Ax|| 7.389179393110e-001 > 7 KSP preconditioned resid norm 4.642286939843e-001 true resid norm 4.642286939846e-001 ||Ae||/||Ax|| 6.668432227160e-001 > 8 KSP preconditioned resid norm 2.225622386414e-001 true resid norm 2.225622386428e-001 ||Ae||/||Ax|| 3.197004459108e-001 > 9 KSP preconditioned resid norm 6.852028979252e-002 true resid norm 6.852028979329e-002 ||Ae||/||Ax|| 9.842625296383e-002 > 10 KSP preconditioned resid norm 2.091417649890e-003 true resid norm 2.091417651046e-003 ||Ae||/||Ax|| 3.004225513287e-003 > 11 KSP preconditioned resid norm 2.221359037659e-005 true resid norm 2.221359037215e-005 ||Ae||/||Ax|| 3.190880353542e-005 > 12 KSP preconditioned resid norm 1.804209918914e-005 true resid norm 1.804209941359e-005 ||Ae||/||Ax|| 2.591664813792e-005 > 13 KSP preconditioned resid norm 1.533714545172e-005 true resid norm 1.533714905817e-005 ||Ae||/||Ax|| 2.203111104021e-005 > 14 KSP preconditioned resid norm 5.645319291054e-008 true resid norm 5.645404908731e-008 ||Ae||/||Ax|| 8.109365172072e-008 > 15 KSP preconditioned resid norm 3.193476220728e-010 true resid norm 3.097685921426e-010 ||Ae||/||Ax|| 4.449683721779e-010 > 16 KSP preconditioned resid norm 3.809088792508e-012 true resid norm 2.536256355736e-011 ||Ae||/||Ax|| 3.643215905885e-011 > 2 SNES Function norm 6.943721478945e-001 > 0 KSP preconditioned resid norm 6.943721478945e-001 true resid norm 6.943721478945e-001 ||Ae||/||Ax|| 1.000000000000e+000 > 1 KSP preconditioned resid norm 6.312241812914e-001 true resid norm 6.312241812914e-001 ||Ae||/||Ax|| 9.090574603338e-001 > 2 KSP preconditioned resid norm 5.826977509711e-001 true resid norm 5.826977509711e-001 ||Ae||/||Ax|| 8.391721251177e-001 > 3 KSP preconditioned resid norm 5.690491242974e-001 true resid norm 5.690491242974e-001 ||Ae||/||Ax|| 8.195160563725e-001 > 4 KSP preconditioned resid norm 5.668884553347e-001 true resid norm 5.668884553347e-001 ||Ae||/||Ax|| 8.164043691178e-001 > 5 KSP preconditioned resid norm 5.667877314429e-001 true resid norm 5.667877314429e-001 ||Ae||/||Ax|| 8.162593116120e-001 > 6 KSP preconditioned resid norm 5.103464760431e-001 true resid norm 5.103464760434e-001 ||Ae||/||Ax|| 7.349754416143e-001 > 7 KSP preconditioned resid norm 4.605661569558e-001 true resid norm 4.605661569565e-001 ||Ae||/||Ax|| 6.632843185791e-001 > 8 KSP preconditioned resid norm 2.208053445289e-001 true resid norm 2.208053445308e-001 ||Ae||/||Ax|| 3.179928014111e-001 > 9 KSP preconditioned resid norm 6.797979448157e-002 true resid norm 6.797979448302e-002 ||Ae||/||Ax|| 9.790109624810e-002 > 10 KSP preconditioned resid norm 2.074808319295e-003 true resid norm 2.074808318610e-003 ||Ae||/||Ax|| 2.988035054259e-003 > 11 KSP preconditioned resid norm 2.203325006674e-005 true resid norm 2.203324811647e-005 ||Ae||/||Ax|| 3.173118072677e-005 > 12 KSP preconditioned resid norm 1.823511305062e-005 true resid norm 1.823511045569e-005 ||Ae||/||Ax|| 2.626129304147e-005 > 13 KSP preconditioned resid norm 1.735630734709e-005 true resid norm 1.735630720647e-005 ||Ae||/||Ax|| 2.499568460385e-005 > 14 KSP preconditioned resid norm 2.741404670294e-008 true resid norm 2.741193558629e-008 ||Ae||/||Ax|| 3.947729710849e-008 > 15 KSP preconditioned resid norm 1.660262277254e-010 true resid norm 1.728810158968e-010 ||Ae||/||Ax|| 2.489745828962e-010 > 16 KSP preconditioned resid norm 4.907918572393e-012 true resid norm 1.668357561843e-011 ||Ae||/||Ax|| 2.402685025461e-011 > 3 SNES Function norm 6.943722078847e-001 > STEP 0 (Newton iterations: 3) > > diverged reason: DIVERGED_LS_FAILURE > > From knepley at gmail.com Tue May 17 08:14:04 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 17 May 2011 08:14:04 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: <4DCBF40F.1020301@berkeley.edu> References: <4DCBF40F.1020301@berkeley.edu> Message-ID: On Thu, May 12, 2011 at 9:51 AM, Sanjay Govindjee wrote: > Sylvain, > > Is there a reason you are not using Prometheus (--download_prometheus=1) > for your MG preconditioner? > It was designed with 3D solid mechanics in mind. I would also point out that even for power law problems, PyLith uses ML and PCFIELDSPLIT for the elastic solver and it works great. Have you tried this? Thanks, Matt > > -sanjay > > > > On 5/11/11 11:12 PM, Sylvain Barbot wrote: > >> Dear Jed, >> >> During my recent visit to ETH, I talked at length about multi-grid >> with Dave May who warned me about the issues of large >> coefficient-contrasts. Most of my problems of interest for >> tectonophysics and earthquake simulations are cases of relatively >> smooth variations in elastic moduli. So I am not too worried about >> this aspect of the problem. I appreciate your advice about trying >> simpler solutions first. I have tested at length direct solvers of 2-D >> and 3-D problems of elastic deformation and I am quite happy with the >> results. My primary concern now is computation speed, especially for >> 3-D problems, where i have of the order 512^3 degrees of freedom. I >> was planning to test Jacobi and SOR smoothers. Is there another >> smoother you recommend for this kind of problem? >> >> Thanks, >> Sylvain >> >> 2011/5/11 Jed Brown: >> >>> On Wed, May 11, 2011 at 04:20, Sylvain Barbot >>> wrote: >>> >>>> I am still trying to design a >>>> multigrid preconditionner for the Navier's equation of elasticity. >>>> >>> I have heard, through an external source, that you have large jumps in >>> both >>> Young's modulus and Poisson ratio that are not grid aligned, including >>> perhaps thin structures that span a large part of the domain. Such >>> problems >>> are pretty hard, so I suggest you focus on robustness and do not worry >>> about >>> low-memory implementation at this point. That is, you should assemble the >>> matrices in a usual PETSc format instead of using MatShell to do >>> everything >>> matrix-free. This gives you access to much stronger smoothers. >>> After you find a scheme that is robust enough for your purposes, _then_ >>> you >>> can make it low-memory by replacing some assembled matrices by MatShell. >>> To >>> realize most of the possible memory savings, it should be sufficient to >>> do >>> this on the finest level only. >>> >> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 17 08:25:20 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 17 May 2011 08:25:20 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: <4DBEAEC3.7020708@geology.wisc.edu> References: <4DBC4DDC.1010604@geology.wisc.edu> <4DBEAEC3.7020708@geology.wisc.edu> Message-ID: On Mon, May 2, 2011 at 8:16 AM, Tabrez Ali wrote: > Is there a way I can use this and other mesh routines from Fortran? The > manual doesn't say much on this. > Yes, but you are right that nothing is in the manual. DMMESH (in petsc-dev) now obeys the full DM interface, so that DMGetMatrix() will return you a properly allocated Mat. So what is the problem? Of course, it is that Petsc has no good way to specify what finite element you are dealing with. The way I was doing this is to encode it using some C++ classes. This turns out to be a bad way to do things. I am currently reworking it so that this information is stored in a simple C struct that you can produce. Should have this done soon. Can you mail me a description of an example you would like to run? Thanks, Matt > > Tabrez > > > On 05/01/2011 09:53 AM, Matthew Knepley wrote: > > On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali wrote: > >> Petsc Developers/Users >> >> I having some performance issues with preallocation in a fully >> unstructured FE code. It would be very helpful if those using FE codes can >> comment. >> >> For a problem of size 100K nodes and 600K tet elements (on 1 cpu) >> >> 1. If I calculate the _exact_ number of non-zeros per row (using a running >> list in Fortran) by looping over nodes & elements, the code takes 17 mins >> (to calculate nnz's/per row, assemble and solve). >> 2. If I dont use a running list and simply get the average of the max >> number of nodes a node might be connected to (again by looping over nodes & >> elements but not using a running list) then it takes 8 mins >> 3. If I just magically guess the right value calculated in 2 and use that >> as average nnz per row then it only takes 25 secs. >> >> Basically in all cases Assembly and Solve are very fast (few seconds) but >> the nnz calculation itself (in 2 and 3) takes a long time. How can this be >> cut down? Is there a heuristic way to estimate the number (as done in 3) >> even if it slightly overestimates the nnz's per row or are efficient ways to >> do step 1 or 2. Right now I have do i=1,num_nodes; do j=1,num_elements ... >> which obviously is slow for large number of nodes/elements. >> > > If you want to see my code doing this, look at > > include/petscdmmesh.hh:preallocateOperatorNew() > > which handles the determination of nonzero structure for a FEM operator. > It should look mostly > like your own code. > > Matt > > >> Thanks in advance >> Tabrez >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Tue May 17 09:36:41 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 17 May 2011 09:36:41 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: References: <4DBC4DDC.1010604@geology.wisc.edu> <4DBEAEC3.7020708@geology.wisc.edu> Message-ID: <4DD287F9.80209@geology.wisc.edu> Matt An example which reads an unstructured mesh (ascii or exodus), partitions it and sets up Mat (with the right preallocation) would be great. If it is in Fortran then that would be perfect. Btw my problem also has Lagrange multiplier constraints. Thanks Tabrez On 05/17/2011 08:25 AM, Matthew Knepley wrote: > On Mon, May 2, 2011 at 8:16 AM, Tabrez Ali > wrote: > > Is there a way I can use this and other mesh routines from > Fortran? The manual doesn't say much on this. > > > Yes, but you are right that nothing is in the manual. DMMESH (in > petsc-dev) now obeys the full DM interface, > so that DMGetMatrix() will return you a properly allocated Mat. So > what is the problem? Of course, it is that > Petsc has no good way to specify what finite element you are dealing with. > > The way I was doing this is to encode it using some C++ classes. This > turns out to be a bad way to do things. > I am currently reworking it so that this information is stored in a > simple C struct that you can produce. Should > have this done soon. > > Can you mail me a description of an example you would like to run? > > Thanks, > > Matt > > > Tabrez > > > On 05/01/2011 09:53 AM, Matthew Knepley wrote: >> On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali >> > wrote: >> >> Petsc Developers/Users >> >> I having some performance issues with preallocation in a >> fully unstructured FE code. It would be very helpful if those >> using FE codes can comment. >> >> For a problem of size 100K nodes and 600K tet elements (on 1 cpu) >> >> 1. If I calculate the _exact_ number of non-zeros per row >> (using a running list in Fortran) by looping over nodes & >> elements, the code takes 17 mins (to calculate nnz's/per row, >> assemble and solve). >> 2. If I dont use a running list and simply get the average of >> the max number of nodes a node might be connected to (again >> by looping over nodes & elements but not using a running >> list) then it takes 8 mins >> 3. If I just magically guess the right value calculated in 2 >> and use that as average nnz per row then it only takes 25 secs. >> >> Basically in all cases Assembly and Solve are very fast (few >> seconds) but the nnz calculation itself (in 2 and 3) takes a >> long time. How can this be cut down? Is there a heuristic way >> to estimate the number (as done in 3) even if it slightly >> overestimates the nnz's per row or are efficient ways to do >> step 1 or 2. Right now I have do i=1,num_nodes; do >> j=1,num_elements ... which obviously is slow for large number >> of nodes/elements. >> >> >> If you want to see my code doing this, look at >> >> include/petscdmmesh.hh:preallocateOperatorNew() >> >> which handles the determination of nonzero structure for a FEM >> operator. It should look mostly >> like your own code. >> >> Matt >> >> Thanks in advance >> Tabrez >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 17 11:21:05 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 17 May 2011 11:21:05 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: <4DD287F9.80209@geology.wisc.edu> References: <4DBC4DDC.1010604@geology.wisc.edu> <4DBEAEC3.7020708@geology.wisc.edu> <4DD287F9.80209@geology.wisc.edu> Message-ID: On Tue, May 17, 2011 at 9:36 AM, Tabrez Ali wrote: > Matt > > An example which reads an unstructured mesh (ascii or exodus), partitions > it and sets up Mat (with the right preallocation) would be great. If it is > in Fortran then that would be perfect. Btw my problem also has Lagrange > multiplier constraints. > Yes, I should have been more specific. What element? Thanks, Matt > Thanks > Tabrez > > > On 05/17/2011 08:25 AM, Matthew Knepley wrote: > > On Mon, May 2, 2011 at 8:16 AM, Tabrez Ali wrote: > >> Is there a way I can use this and other mesh routines from Fortran? The >> manual doesn't say much on this. >> > > Yes, but you are right that nothing is in the manual. DMMESH (in > petsc-dev) now obeys the full DM interface, > so that DMGetMatrix() will return you a properly allocated Mat. So what is > the problem? Of course, it is that > Petsc has no good way to specify what finite element you are dealing with. > > The way I was doing this is to encode it using some C++ classes. This > turns out to be a bad way to do things. > I am currently reworking it so that this information is stored in a simple > C struct that you can produce. Should > have this done soon. > > Can you mail me a description of an example you would like to run? > > Thanks, > > Matt > > >> >> Tabrez >> >> >> On 05/01/2011 09:53 AM, Matthew Knepley wrote: >> >> On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali wrote: >> >>> Petsc Developers/Users >>> >>> I having some performance issues with preallocation in a fully >>> unstructured FE code. It would be very helpful if those using FE codes can >>> comment. >>> >>> For a problem of size 100K nodes and 600K tet elements (on 1 cpu) >>> >>> 1. If I calculate the _exact_ number of non-zeros per row (using a >>> running list in Fortran) by looping over nodes & elements, the code takes 17 >>> mins (to calculate nnz's/per row, assemble and solve). >>> 2. If I dont use a running list and simply get the average of the max >>> number of nodes a node might be connected to (again by looping over nodes & >>> elements but not using a running list) then it takes 8 mins >>> 3. If I just magically guess the right value calculated in 2 and use that >>> as average nnz per row then it only takes 25 secs. >>> >>> Basically in all cases Assembly and Solve are very fast (few seconds) but >>> the nnz calculation itself (in 2 and 3) takes a long time. How can this be >>> cut down? Is there a heuristic way to estimate the number (as done in 3) >>> even if it slightly overestimates the nnz's per row or are efficient ways to >>> do step 1 or 2. Right now I have do i=1,num_nodes; do j=1,num_elements ... >>> which obviously is slow for large number of nodes/elements. >>> >> >> If you want to see my code doing this, look at >> >> include/petscdmmesh.hh:preallocateOperatorNew() >> >> which handles the determination of nonzero structure for a FEM operator. >> It should look mostly >> like your own code. >> >> Matt >> >> >>> Thanks in advance >>> Tabrez >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Tue May 17 11:53:31 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 17 May 2011 11:53:31 -0500 Subject: [petsc-users] Preallocation (Unstructured FE) In-Reply-To: References: <4DBC4DDC.1010604@geology.wisc.edu> <4DBEAEC3.7020708@geology.wisc.edu> <4DD287F9.80209@geology.wisc.edu> Message-ID: <4DD2A80B.2000505@geology.wisc.edu> Matt Any linear 2/3d element would do. A linear tetrahedron would be good to start with. Thanks Tabrez On 05/17/2011 11:21 AM, Matthew Knepley wrote: > On Tue, May 17, 2011 at 9:36 AM, Tabrez Ali > wrote: > > Matt > > An example which reads an unstructured mesh (ascii or exodus), > partitions it and sets up Mat (with the right preallocation) would > be great. If it is in Fortran then that would be perfect. Btw my > problem also has Lagrange multiplier constraints. > > > Yes, I should have been more specific. What element? > > Thanks, > > Matt > > Thanks > Tabrez > > > On 05/17/2011 08:25 AM, Matthew Knepley wrote: >> On Mon, May 2, 2011 at 8:16 AM, Tabrez Ali >> > wrote: >> >> Is there a way I can use this and other mesh routines from >> Fortran? The manual doesn't say much on this. >> >> >> Yes, but you are right that nothing is in the manual. DMMESH (in >> petsc-dev) now obeys the full DM interface, >> so that DMGetMatrix() will return you a properly allocated Mat. >> So what is the problem? Of course, it is that >> Petsc has no good way to specify what finite element you are >> dealing with. >> >> The way I was doing this is to encode it using some C++ classes. >> This turns out to be a bad way to do things. >> I am currently reworking it so that this information is stored in >> a simple C struct that you can produce. Should >> have this done soon. >> >> Can you mail me a description of an example you would like to run? >> >> Thanks, >> >> Matt >> >> >> Tabrez >> >> >> On 05/01/2011 09:53 AM, Matthew Knepley wrote: >>> On Sat, Apr 30, 2011 at 12:58 PM, Tabrez Ali >>> > wrote: >>> >>> Petsc Developers/Users >>> >>> I having some performance issues with preallocation in a >>> fully unstructured FE code. It would be very helpful if >>> those using FE codes can comment. >>> >>> For a problem of size 100K nodes and 600K tet elements >>> (on 1 cpu) >>> >>> 1. If I calculate the _exact_ number of non-zeros per >>> row (using a running list in Fortran) by looping over >>> nodes & elements, the code takes 17 mins (to calculate >>> nnz's/per row, assemble and solve). >>> 2. If I dont use a running list and simply get the >>> average of the max number of nodes a node might be >>> connected to (again by looping over nodes & elements but >>> not using a running list) then it takes 8 mins >>> 3. If I just magically guess the right value calculated >>> in 2 and use that as average nnz per row then it only >>> takes 25 secs. >>> >>> Basically in all cases Assembly and Solve are very fast >>> (few seconds) but the nnz calculation itself (in 2 and >>> 3) takes a long time. How can this be cut down? Is there >>> a heuristic way to estimate the number (as done in 3) >>> even if it slightly overestimates the nnz's per row or >>> are efficient ways to do step 1 or 2. Right now I have >>> do i=1,num_nodes; do j=1,num_elements ... which >>> obviously is slow for large number of nodes/elements. >>> >>> >>> If you want to see my code doing this, look at >>> >>> include/petscdmmesh.hh:preallocateOperatorNew() >>> >>> which handles the determination of nonzero structure for a >>> FEM operator. It should look mostly >>> like your own code. >>> >>> Matt >>> >>> Thanks in advance >>> Tabrez >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin >>> their experiments is infinitely more interesting than any >>> results to which their experiments lead. >>> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylbar.vainbot at gmail.com Tue May 17 13:32:02 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Tue, 17 May 2011 11:32:02 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: Hi Jed, You're right, I should experiment more with assembled matrices before choosing a solver/preconditioner. Could you be more specific about the command line options that I could test. I have a version of the code with assembled sparse matrices (default format). I tried running with the option -pc_type mg and got the following error: [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] PCSetFromOptions_MG line 318 src/ksp/pc/impls/mg/mg.c [0]PETSC ERROR: [0] PCSetFromOptions line 166 src/ksp/pc/interface/pcset.c [0]PETSC ERROR: [0] KSPSetFromOptions line 320 src/ksp/ksp/interface/itcl.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 14:42:02 CDT 2010 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./planestrain on a darwin10. named catalina.gps.caltech.edu by sbarbot Tue May 17 11:24:14 2011 [0]PETSC ERROR: Libraries linked from /Users/sbarbot/Documents/work/petsc/darwin10.4.0-c-debug/lib [0]PETSC ERROR: Configure run at Mon Aug 2 17:15:50 2010 [0]PETSC ERROR: Configure options --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t --with-mpi-dir=/shared/bin/ [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0 in the code, the matrix assembly for c%A is followed by: CALL KSPCreate(PETSC_COMM_WORLD,c%ksp,ierr) CALL KSPSetOperators(c%ksp,c%A,c%A,DIFFERENT_NONZERO_PATTERN,ierr) CALL KSPSetFromOptions(c%ksp,ierr) CALL KSPSetUp(c%ksp,ierr) Thanks for your suggestions, Sylvain 2011/5/12 Jed Brown : > On Thu, May 12, 2011 at 08:12, Sylvain Barbot > wrote: >> >> My primary concern now is computation speed, especially for >> 3-D problems, where i have of the order 512^3 degrees of freedom. I >> was planning to test Jacobi and SOR smoothers. Is there another >> smoother you recommend for this kind of problem? > > It's worth trying ILU as a smoother, especially if you are using a stretched > mesh. What discretization are you using? I still recommend just writing the > matrix assembly and experimenting (you can do everything on the command > line). Then, if the problem of interest responds well to SOR, you can easily > put in an option to use matrix-free. That is, you would have one code with a > runtime option of which levels to assemble versus handle matrix-free. The > fastest choice will almost certainly depend on the problem and on the > hardware that you are running it on, so it's well worth having both options > available. From jed at 59A2.org Tue May 17 13:56:52 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 17 May 2011 20:56:52 +0200 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: Message-ID: On Tue, May 17, 2011 at 20:32, Sylvain Barbot wrote: > You're right, I should experiment more with assembled matrices before > choosing a solver/preconditioner. Could you be more specific about the > command line options that I could test. I have a version of the code > with assembled sparse matrices (default format). I tried running with > the option -pc_type mg and got the following error: > You need to set grid information or interpolation operators for this to work. If you have a grid hierarchy, you can set it by following that chapter of the user's manual (which I think you were trying to do with matrix-free operators everywhere). If you use a structured grid, DMMG is quite convenient. (petsc-dev does not seg-fault on the configuration you gave because the data structure has been improved) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylbar.vainbot at gmail.com Tue May 17 14:23:45 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Tue, 17 May 2011 12:23:45 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> Message-ID: Dear Barry, I made your suggested changes and recompiled the library. I also added the function ourview and ourgetvecs from petsc-dev and changed the last call of matshellsetoperations_ from PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL,"Cannot set that matrix operation"); to PetscError(__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,0,1,"Cannot set that matrix operation"); for compatibility with my current version of Petsc. I changed the matrix type of my smoothing matrix to MAT_SOR. Upon running, i get the following error [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: This matrix type, shell, does not support getting diagonal block! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 14:42:02 CDT 2010 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named catalina.gps.caltech.edu by sbarbot Tue May 17 12:08:10 2011 [0]PETSC ERROR: Libraries linked from /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib [0]PETSC ERROR: Configure run at Thu Oct 7 19:12:09 2010 [0]PETSC ERROR: Configure options --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single --with-petsc-arch=real4-with-debug [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c btw, when I call MatShellSetOperation, what should be the type of the shell matrix that performs the residuals, MAT_SOR as well, MAT_MULT or something else? thanks again for your help. Sylvain 2011/5/16 Barry Smith : > > On May 16, 2011, at 9:52 PM, Sylvain Barbot wrote: > >> Hi Barry, >> >> Thanks for your comments. >> >>>> ? ? ? CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); >>> >>> ? ?You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. >> >> My function matrixsmoothfv indeed performs the smoothing operation so >> it not a multiplication. I can see no documentation about MATOP_SOR, >> is there an example somewhere online? > > ? For each possible Matrix function there is the name, for matrix vector multiply it is MATOP_MULT and for smoothing it is MATOP_SOR and the calling sequence you use in your function that does the operation is the same as the matrix operation. For example for MatMult() it is Mat,Vec,Vec,PetscErrorCode (see below for the smoother). >> Assigning MATOP_SOR to my >> smoothing matrix now generates the following runtime error message: >> >> [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in >> src/mat/impls/shell/ftn-custom/zshellf.c >> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 >> rank 0 in job 37 ?catalina.gps.caltech.edu_50093 ? caused collective >> abort of all ranks >> ?exit status of rank 0: return code 1 > > ? ?This is because we hadn't added support for providing the MATOP_SOR for the Fortran interface. Edit src/mat/imples/shell/ftn-custom/zshellf.c and locate the function matshellsetoperation_() and put instead of it the following code. > > > static PetscErrorCode oursor(Mat mat,Vec b,PetscReal omega,MatSORType flg,PetscReal shift,PetscInt its,PetscInt lits,Vec x) > { > ?PetscErrorCode ierr = 0; > ?(*(PetscErrorCode (PETSC_STDCALL *)(Mat*,Vec*,PetscReal*,MatSORType*,PetscReal*,PetscInt*,PetscInt*,Vec*,PetscErrorCode*))(((PetscObject)mat)->fortran_func_pointers[9]))(&mat,&b,&omega,&flg,&shift,&its,&lits,&x,&ierr); > ?return ierr; > } > > void PETSC_STDCALL matshellsetoperation_(Mat *mat,MatOperation *op,PetscErrorCode (PETSC_STDCALL *f)(Mat*,Vec*,Vec*,PetscErrorCode*),PetscErrorCode *ierr) > { > ?MPI_Comm comm; > > ?*ierr = PetscObjectGetComm((PetscObject)*mat,&comm);if (*ierr) return; > ?PetscObjectAllocateFortranPointers(*mat,10); > ?if (*op == MATOP_MULT) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmult); > ? ?((PetscObject)*mat)->fortran_func_pointers[0] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_MULT_TRANSPOSE) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttranspose); > ? ?((PetscObject)*mat)->fortran_func_pointers[2] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_MULT_ADD) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmultadd); > ? ?((PetscObject)*mat)->fortran_func_pointers[1] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_MULT_TRANSPOSE_ADD) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttransposeadd); > ? ?((PetscObject)*mat)->fortran_func_pointers[3] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_GET_DIAGONAL) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetdiagonal); > ? ?((PetscObject)*mat)->fortran_func_pointers[4] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_DIAGONAL_SCALE) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalscale); > ? ?((PetscObject)*mat)->fortran_func_pointers[5] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_DIAGONAL_SET) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalset); > ? ?((PetscObject)*mat)->fortran_func_pointers[6] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_GET_VECS) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetvecs); > ? ?((PetscObject)*mat)->fortran_func_pointers[7] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_VIEW) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourview); > ? ?((PetscObject)*mat)->fortran_func_pointers[8] = (PetscVoidFunction)f; > ?} else if (*op == MATOP_SOR) { > ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)oursor); > ? ?((PetscObject)*mat)->fortran_func_pointers[9] = (PetscVoidFunction)f; > ?} else { > ? ?PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL, > ? ? ? ? ? ? ? "Cannot set that matrix operation"); > ? ?*ierr = 1; > ?} > } > > > Then in that directory run make to update the PETSc libraries. > > Now note the calling sequence of the relaxation function you need to provide. It is PetscErrorCode ?MatSOR(Mat mat,Vec b,PetscReal omega,MatSORType flag,PetscReal shift,PetscInt its,PetscInt lits,Vec x,PetscErrorCode ierr) > in your function you can ignore the omega, the flag, and the shift argument. You function should do its*lits smoothing steps. > > Once you have this going please let us know what happens. > > ?Barry > > > > >> >>>> I also provide work space... >>>> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >>>> ERROR: PCSetUp_BJacobi()". >>> >>> It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. ?What solver options are you running with? >> >> I setup the solver with: >> >> CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); >> >> but this error might not be representative of the problem. upstream, >> my smoothing matrices were not set up properly. >> >>>> MatGetVecs() line 7265 in src/mat/interface/matrix.c >>>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>>> [0]PETSC ERROR: KSPSetUp() line >>>> >>> ? ? This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. > > From bsmith at mcs.anl.gov Tue May 17 14:44:07 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 17 May 2011 14:44:07 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> Message-ID: <4FB1A2D2-1A4D-454C-9A72-A89795C209A2@mcs.anl.gov> This problem is because it is trying to use block Jacobi on a level as the smoother instead of SOR. You need to tell PCMG to use sor with -mg_levels_pc_type sor and make sure you have a KSPSetFromOptions() call before KSPSolve() so it catches those options. I'm sorry I forgot about the ourgetvecs() you did the correct fix. I will add that to petsc-dev also. Barry The default is block Jacobi with ILU on each process. On May 17, 2011, at 2:23 PM, Sylvain Barbot wrote: > Dear Barry, > > I made your suggested changes and recompiled the library. I also added > the function ourview and ourgetvecs from petsc-dev and changed the > last call of matshellsetoperations_ from > > PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL,"Cannot > set that matrix operation"); > > to > > PetscError(__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,0,1,"Cannot > set that matrix operation"); > > for compatibility with my current version of Petsc. > > I changed the matrix type of my smoothing matrix to MAT_SOR. Upon > running, i get the following error > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: This matrix type, shell, does not support getting > diagonal block! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 > 14:42:02 CDT 2010 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named > catalina.gps.caltech.edu by sbarbot Tue May 17 12:08:10 2011 > [0]PETSC ERROR: Libraries linked from > /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib > [0]PETSC ERROR: Configure run at Thu Oct 7 19:12:09 2010 > [0]PETSC ERROR: Configure options > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t > -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single > --with-petsc-arch=real4-with-debug > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > > btw, when I call MatShellSetOperation, what should be the type of the > shell matrix that performs the residuals, MAT_SOR as well, MAT_MULT or > something else? > > thanks again for your help. > Sylvain > > 2011/5/16 Barry Smith : >> >> On May 16, 2011, at 9:52 PM, Sylvain Barbot wrote: >> >>> Hi Barry, >>> >>> Thanks for your comments. >>> >>>>> CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); >>>> >>>> You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. >>> >>> My function matrixsmoothfv indeed performs the smoothing operation so >>> it not a multiplication. I can see no documentation about MATOP_SOR, >>> is there an example somewhere online? >> >> For each possible Matrix function there is the name, for matrix vector multiply it is MATOP_MULT and for smoothing it is MATOP_SOR and the calling sequence you use in your function that does the operation is the same as the matrix operation. For example for MatMult() it is Mat,Vec,Vec,PetscErrorCode (see below for the smoother). >>> Assigning MATOP_SOR to my >>> smoothing matrix now generates the following runtime error message: >>> >>> [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in >>> src/mat/impls/shell/ftn-custom/zshellf.c >>> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 >>> rank 0 in job 37 catalina.gps.caltech.edu_50093 caused collective >>> abort of all ranks >>> exit status of rank 0: return code 1 >> >> This is because we hadn't added support for providing the MATOP_SOR for the Fortran interface. Edit src/mat/imples/shell/ftn-custom/zshellf.c and locate the function matshellsetoperation_() and put instead of it the following code. >> >> >> static PetscErrorCode oursor(Mat mat,Vec b,PetscReal omega,MatSORType flg,PetscReal shift,PetscInt its,PetscInt lits,Vec x) >> { >> PetscErrorCode ierr = 0; >> (*(PetscErrorCode (PETSC_STDCALL *)(Mat*,Vec*,PetscReal*,MatSORType*,PetscReal*,PetscInt*,PetscInt*,Vec*,PetscErrorCode*))(((PetscObject)mat)->fortran_func_pointers[9]))(&mat,&b,&omega,&flg,&shift,&its,&lits,&x,&ierr); >> return ierr; >> } >> >> void PETSC_STDCALL matshellsetoperation_(Mat *mat,MatOperation *op,PetscErrorCode (PETSC_STDCALL *f)(Mat*,Vec*,Vec*,PetscErrorCode*),PetscErrorCode *ierr) >> { >> MPI_Comm comm; >> >> *ierr = PetscObjectGetComm((PetscObject)*mat,&comm);if (*ierr) return; >> PetscObjectAllocateFortranPointers(*mat,10); >> if (*op == MATOP_MULT) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmult); >> ((PetscObject)*mat)->fortran_func_pointers[0] = (PetscVoidFunction)f; >> } else if (*op == MATOP_MULT_TRANSPOSE) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttranspose); >> ((PetscObject)*mat)->fortran_func_pointers[2] = (PetscVoidFunction)f; >> } else if (*op == MATOP_MULT_ADD) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmultadd); >> ((PetscObject)*mat)->fortran_func_pointers[1] = (PetscVoidFunction)f; >> } else if (*op == MATOP_MULT_TRANSPOSE_ADD) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttransposeadd); >> ((PetscObject)*mat)->fortran_func_pointers[3] = (PetscVoidFunction)f; >> } else if (*op == MATOP_GET_DIAGONAL) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetdiagonal); >> ((PetscObject)*mat)->fortran_func_pointers[4] = (PetscVoidFunction)f; >> } else if (*op == MATOP_DIAGONAL_SCALE) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalscale); >> ((PetscObject)*mat)->fortran_func_pointers[5] = (PetscVoidFunction)f; >> } else if (*op == MATOP_DIAGONAL_SET) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalset); >> ((PetscObject)*mat)->fortran_func_pointers[6] = (PetscVoidFunction)f; >> } else if (*op == MATOP_GET_VECS) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetvecs); >> ((PetscObject)*mat)->fortran_func_pointers[7] = (PetscVoidFunction)f; >> } else if (*op == MATOP_VIEW) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourview); >> ((PetscObject)*mat)->fortran_func_pointers[8] = (PetscVoidFunction)f; >> } else if (*op == MATOP_SOR) { >> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)oursor); >> ((PetscObject)*mat)->fortran_func_pointers[9] = (PetscVoidFunction)f; >> } else { >> PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL, >> "Cannot set that matrix operation"); >> *ierr = 1; >> } >> } >> >> >> Then in that directory run make to update the PETSc libraries. >> >> Now note the calling sequence of the relaxation function you need to provide. It is PetscErrorCode MatSOR(Mat mat,Vec b,PetscReal omega,MatSORType flag,PetscReal shift,PetscInt its,PetscInt lits,Vec x,PetscErrorCode ierr) >> in your function you can ignore the omega, the flag, and the shift argument. You function should do its*lits smoothing steps. >> >> Once you have this going please let us know what happens. >> >> Barry >> >> >> >> >>> >>>>> I also provide work space... >>>>> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >>>>> ERROR: PCSetUp_BJacobi()". >>>> >>>> It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. What solver options are you running with? >>> >>> I setup the solver with: >>> >>> CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); >>> >>> but this error might not be representative of the problem. upstream, >>> my smoothing matrices were not set up properly. >>> >>>>> MatGetVecs() line 7265 in src/mat/interface/matrix.c >>>>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>>>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>>>> [0]PETSC ERROR: KSPSetUp() line >>>>> >>>> This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. >> >> From gshy2014 at gmail.com Tue May 17 17:26:51 2011 From: gshy2014 at gmail.com (Shiyuan) Date: Tue, 17 May 2011 17:26:51 -0500 Subject: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? Message-ID: Is there any support in PETSC (with other packages) for finite element multigrid on unstructured meshes? Any recommended way to do that? Any tutorial or examples? Thanks. ------------------------------ Message: 3 Date: Mon, 16 May 2011 14:08:12 -0500 From: Barry Smith Subject: Re: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? To: PETSc users list Message-ID: Content-Type: text/plain; charset=us-ascii On May 16, 2011, at 1:36 AM, Li, Zhisong (lizs) wrote: > Hi, Petsc Team, > > In my knowledge of Petsc, the PCMG object is the main tool for general multigrid calculations, as DMMG only works for simple constant matrix and RHS. This is not correct. DMMG can be used for non-constant matrix. DMMG is restricted to structured grids only. But if your problem is on a two or three dimensional structured grid you can use it and it is much easier than coding the pieces directly. > But it's difficult to find any example code for using PCMG. I checked out the petsc-user mail archive on this topic but still not find a good description on it. I encounter similar errors in the practice as some previous question-raisers did. > > Here is my simple code implementing the PCMG based on my understanding from the user manual and tutorial. It is supposed to have multigrid of only two levels (finest and coarsest grid). The error message indicates SEGV segmentation violation because of this part of code. Anything wrong with this implementation? > This code looks more or less correct. You need to run with -start_in_debugger to track down what is trigger the SEGV src/ksp/ksp/examples/ tutorials/ex19.c is a simple example that runs on two levels. Barry > > DAGetMatrix(da, MATAIJ, &M); > > KSPCreate(PETSC_COMM_WORLD, &ksp); > KSPSetType(ksp, KSPGMRES); > KSPGetPC(ksp, &pcmg); > PCSetType(pcmg, PCMG); > > PCMGSetLevels(pcmg, 2, &PETSC_COMM_WORLD); > PCMGSetType(pcmg, PC_MG_MULTIPLICATIVE); > PCMGSetCycleType(pcmg, PC_MG_CYCLE_W); > PCMGSetCyclesOnLevel(pcmg, 0, 1); > PCMGSetCyclesOnLevel(pcmg, 1, 1); > > PCMGGetCoarseSolve(pcmg, &ksp); > > PCMGGetSmoother(pcmg, 0, &ksp); > PCMGGetSmoother(pcmg, 1, &ksp); > PCMGSetInterpolation(pcmg, 1, M); > PCMGSetRestriction(pcmg, 1, M); > > PCMGSetResidual(pcmg, 0, PCMGDefaultResidual, M); > PCMGSetResidual(pcmg, 1, PCMGDefaultResidual, M); > > > The mandatory remaining PCMGSetRhs( ), PCMGSetX( ) and PCMGSetR( ) functions should be generated by Petsc automatically. > > It is tough to learn some Petsc functions when no detailed example and few guidance is provided. I wish you can add some tutorials codes on PCMG usage in the future version of Petsc. > > > Thank you very much. > > > Zhisong Li > > > ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From austin at txcorp.com Tue May 17 17:31:04 2011 From: austin at txcorp.com (Travis Austin) Date: Tue, 17 May 2011 15:31:04 -0700 Subject: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? In-Reply-To: References: Message-ID: <831EBE53-0788-4836-9424-51FA0DD8D080@txcorp.com> You would most likely want to use an Algebraic Multigrid method. See HYPRE's Boomeramg or ML. Both can be downloaded through the configure line when you configure PETSc. Of course the performance is going to depend on what type of equations you are solving. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Travis Austin, Ph.D. Tech-X Corporation 5621 Arapahoe Ave, Suite A Boulder, CO 80303 austin at txcorp.com ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On May 17, 2011, at 3:26 PM, Shiyuan wrote: > Is there any support in PETSC (with other packages) for finite element multigrid on unstructured meshes? Any recommended way to do that? Any tutorial or examples? Thanks. > > ------------------------------ > > Message: 3 > Date: Mon, 16 May 2011 14:08:12 -0500 > From: Barry Smith > Subject: Re: [petsc-users] Can you provide a basic example of using > PCMG for multigrid applications? > To: PETSc users list > Message-ID: > Content-Type: text/plain; charset=us-ascii > > > On May 16, 2011, at 1:36 AM, Li, Zhisong (lizs) wrote: > > > Hi, Petsc Team, > > > > In my knowledge of Petsc, the PCMG object is the main tool for general multigrid calculations, as DMMG only works for simple constant matrix and RHS. > > This is not correct. DMMG can be used for non-constant matrix. DMMG is restricted to structured grids only. But if your problem is on a two or three dimensional structured grid you can use it and it is much easier than coding the pieces directly. > > > > > But it's difficult to find any example code for using PCMG. I checked out the petsc-user mail archive on this topic but still not find a good description on it. I encounter similar errors in the practice as some previous question-raisers did. > > > > Here is my simple code implementing the PCMG based on my understanding from the user manual and tutorial. It is supposed to have multigrid of only two levels (finest and coarsest grid). The error message indicates SEGV segmentation violation because of this part of code. Anything wrong with this implementation? > > > > This code looks more or less correct. You need to run with -start_in_debugger to track down what is trigger the SEGV > > src/ksp/ksp/examples/ > tutorials/ex19.c is a simple example that runs on two levels. > > > Barry > > > > > > DAGetMatrix(da, MATAIJ, &M); > > > > KSPCreate(PETSC_COMM_WORLD, &ksp); > > KSPSetType(ksp, KSPGMRES); > > KSPGetPC(ksp, &pcmg); > > PCSetType(pcmg, PCMG); > > > > PCMGSetLevels(pcmg, 2, &PETSC_COMM_WORLD); > > PCMGSetType(pcmg, PC_MG_MULTIPLICATIVE); > > PCMGSetCycleType(pcmg, PC_MG_CYCLE_W); > > PCMGSetCyclesOnLevel(pcmg, 0, 1); > > PCMGSetCyclesOnLevel(pcmg, 1, 1); > > > > PCMGGetCoarseSolve(pcmg, &ksp); > > > > PCMGGetSmoother(pcmg, 0, &ksp); > > PCMGGetSmoother(pcmg, 1, &ksp); > > PCMGSetInterpolation(pcmg, 1, M); > > PCMGSetRestriction(pcmg, 1, M); > > > > PCMGSetResidual(pcmg, 0, PCMGDefaultResidual, M); > > PCMGSetResidual(pcmg, 1, PCMGDefaultResidual, M); > > > > > > The mandatory remaining PCMGSetRhs( ), PCMGSetX( ) and PCMGSetR( ) functions should be generated by Petsc automatically. > > > > It is tough to learn some Petsc functions when no detailed example and few guidance is provided. I wish you can add some tutorials codes on PCMG usage in the future version of Petsc. > > > > > > Thank you very much. > > > > > > Zhisong Li > > > > > > > > > > ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylbar.vainbot at gmail.com Tue May 17 19:39:42 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Tue, 17 May 2011 17:39:42 -0700 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: <4FB1A2D2-1A4D-454C-9A72-A89795C209A2@mcs.anl.gov> References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> <4FB1A2D2-1A4D-454C-9A72-A89795C209A2@mcs.anl.gov> Message-ID: Barry, I have a call to KSPSetFromOptions() before KSPSetup() and KSPSolve() already. Is there any way I can force the default to be SOR smoother in the code instead of from the command line? With the command line you suggested, I get a slightly different error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Object is in wrong state! [0]PETSC ERROR: Matrix must be set first! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 14:42:02 CDT 2010 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named catalina.gps.caltech.edu by sbarbot Tue May 17 17:30:17 2011 [0]PETSC ERROR: Libraries linked from /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib [0]PETSC ERROR: Configure run at Thu Oct 7 19:12:09 2010 [0]PETSC ERROR: Configure options --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single --with-petsc-arch=real4-with-debug [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCSetUp() line 775 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: PCSetUp_MG() line 602 in src/ksp/pc/impls/mg/mg.c [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c there is no more reference to PCSetUp_BJacobi(), so it's a half success. I attach the code setting up the solver/preconditioner for reference (SUBROUTINE green_init_mg). btw, is there a way to better trace the location of the problem? i am using the __FUNCT__ pre-compiling definition, for instance #undef __FUNCT__ #define __FUNCT__ "matrixinterp" SUBROUTINE matrixinterp(A,x,y,ierr) but it never shows up in the warning/error message. Thanks, Sylvain 2011/5/17 Barry Smith : > > ? This problem is because it is trying to use block Jacobi on a level as the smoother instead of SOR. You need to tell PCMG to use sor with -mg_levels_pc_type sor ? and make sure you have a KSPSetFromOptions() call before KSPSolve() so it catches those options. > > ? I'm sorry I forgot about the ourgetvecs() ?you did the correct fix. I will add that to petsc-dev also. > > > ? Barry > > The default is block Jacobi with ILU on each process. > > > On May 17, 2011, at 2:23 PM, Sylvain Barbot wrote: > >> Dear Barry, >> >> I made your suggested changes and recompiled the library. I also added >> the function ourview and ourgetvecs from petsc-dev and changed the >> last call of matshellsetoperations_ from >> >> PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL,"Cannot >> set that matrix operation"); >> >> to >> >> PetscError(__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,0,1,"Cannot >> set that matrix operation"); >> >> for compatibility with my current version of Petsc. >> >> I changed the matrix type of my smoothing matrix to MAT_SOR. Upon >> running, i get the following error >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: No support for this operation for this object type! >> [0]PETSC ERROR: This matrix type, shell, does not support getting >> diagonal block! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 >> 14:42:02 CDT 2010 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named >> catalina.gps.caltech.edu by sbarbot Tue May 17 12:08:10 2011 >> [0]PETSC ERROR: Libraries linked from >> /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib >> [0]PETSC ERROR: Configure run at Thu Oct ?7 19:12:09 2010 >> [0]PETSC ERROR: Configure options >> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t >> -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single >> --with-petsc-arch=real4-with-debug >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c >> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c >> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >> >> btw, when I call MatShellSetOperation, what should be the type of the >> shell matrix that performs the residuals, MAT_SOR as well, MAT_MULT or >> something else? >> >> thanks again for your help. >> Sylvain >> >> 2011/5/16 Barry Smith : >>> >>> On May 16, 2011, at 9:52 PM, Sylvain Barbot wrote: >>> >>>> Hi Barry, >>>> >>>> Thanks for your comments. >>>> >>>>>> ? ? ? CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); >>>>> >>>>> ? ?You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. >>>> >>>> My function matrixsmoothfv indeed performs the smoothing operation so >>>> it not a multiplication. I can see no documentation about MATOP_SOR, >>>> is there an example somewhere online? >>> >>> ? For each possible Matrix function there is the name, for matrix vector multiply it is MATOP_MULT and for smoothing it is MATOP_SOR and the calling sequence you use in your function that does the operation is the same as the matrix operation. For example for MatMult() it is Mat,Vec,Vec,PetscErrorCode (see below for the smoother). >>>> Assigning MATOP_SOR to my >>>> smoothing matrix now generates the following runtime error message: >>>> >>>> [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in >>>> src/mat/impls/shell/ftn-custom/zshellf.c >>>> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 >>>> rank 0 in job 37 ?catalina.gps.caltech.edu_50093 ? caused collective >>>> abort of all ranks >>>> ?exit status of rank 0: return code 1 >>> >>> ? ?This is because we hadn't added support for providing the MATOP_SOR for the Fortran interface. Edit src/mat/imples/shell/ftn-custom/zshellf.c and locate the function matshellsetoperation_() and put instead of it the following code. >>> >>> >>> static PetscErrorCode oursor(Mat mat,Vec b,PetscReal omega,MatSORType flg,PetscReal shift,PetscInt its,PetscInt lits,Vec x) >>> { >>> ?PetscErrorCode ierr = 0; >>> ?(*(PetscErrorCode (PETSC_STDCALL *)(Mat*,Vec*,PetscReal*,MatSORType*,PetscReal*,PetscInt*,PetscInt*,Vec*,PetscErrorCode*))(((PetscObject)mat)->fortran_func_pointers[9]))(&mat,&b,&omega,&flg,&shift,&its,&lits,&x,&ierr); >>> ?return ierr; >>> } >>> >>> void PETSC_STDCALL matshellsetoperation_(Mat *mat,MatOperation *op,PetscErrorCode (PETSC_STDCALL *f)(Mat*,Vec*,Vec*,PetscErrorCode*),PetscErrorCode *ierr) >>> { >>> ?MPI_Comm comm; >>> >>> ?*ierr = PetscObjectGetComm((PetscObject)*mat,&comm);if (*ierr) return; >>> ?PetscObjectAllocateFortranPointers(*mat,10); >>> ?if (*op == MATOP_MULT) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmult); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[0] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_MULT_TRANSPOSE) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttranspose); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[2] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_MULT_ADD) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmultadd); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[1] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_MULT_TRANSPOSE_ADD) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttransposeadd); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[3] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_GET_DIAGONAL) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetdiagonal); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[4] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_DIAGONAL_SCALE) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalscale); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[5] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_DIAGONAL_SET) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalset); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[6] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_GET_VECS) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetvecs); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[7] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_VIEW) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourview); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[8] = (PetscVoidFunction)f; >>> ?} else if (*op == MATOP_SOR) { >>> ? ?*ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)oursor); >>> ? ?((PetscObject)*mat)->fortran_func_pointers[9] = (PetscVoidFunction)f; >>> ?} else { >>> ? ?PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL, >>> ? ? ? ? ? ? ? "Cannot set that matrix operation"); >>> ? ?*ierr = 1; >>> ?} >>> } >>> >>> >>> Then in that directory run make to update the PETSc libraries. >>> >>> Now note the calling sequence of the relaxation function you need to provide. It is PetscErrorCode ?MatSOR(Mat mat,Vec b,PetscReal omega,MatSORType flag,PetscReal shift,PetscInt its,PetscInt lits,Vec x,PetscErrorCode ierr) >>> in your function you can ignore the omega, the flag, and the shift argument. You function should do its*lits smoothing steps. >>> >>> Once you have this going please let us know what happens. >>> >>> ?Barry >>> >>> >>> >>> >>>> >>>>>> I also provide work space... >>>>>> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >>>>>> ERROR: PCSetUp_BJacobi()". >>>>> >>>>> It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. ?What solver options are you running with? >>>> >>>> I setup the solver with: >>>> >>>> CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); >>>> >>>> but this error might not be representative of the problem. upstream, >>>> my smoothing matrices were not set up properly. >>>> >>>>>> MatGetVecs() line 7265 in src/mat/interface/matrix.c >>>>>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>>>>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>>>>> [0]PETSC ERROR: KSPSetUp() line >>>>>> >>>>> ? ? This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. >>> >>> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: green.f90 Type: application/octet-stream Size: 45011 bytes Desc: not available URL: From sylbar.vainbot at gmail.com Tue May 17 19:53:35 2011 From: sylbar.vainbot at gmail.com (Sylvain Barbot) Date: Tue, 17 May 2011 17:53:35 -0700 Subject: [petsc-users] Can you provide a basic example of using PCMG for multigrid applications? In-Reply-To: <88D7E3BB7E1960428303E760100374515F13A729@BL2PRD0103MB055.prod.exchangelabs.com> References: <88D7E3BB7E1960428303E760100374515F13A729@BL2PRD0103MB055.prod.exchangelabs.com> Message-ID: Zhisong, It looks like you're using the same ksp object to define your solver/preconditioner, the smoother and the coarse-level solver. It looks like a recursive call to your solver. Perhaps you should use a different structure for the coarse-level solver: PCMGGetCoarseSolve(pcmg, &coarseksp); and define coarseksp entirely for the coarse level. Similarly, you may need to define a specific ksp for the smoothers on each level. Cheers, Sylvain 2011/5/15 Li, Zhisong (lizs) : > ?Hi, Petsc Team, > > In my knowledge of Petsc, the PCMG object is the main tool for general > multigrid calculations, as DMMG only works for simple constant matrix and > RHS.? But it's difficult to find any example code for using PCMG.? I checked > out the petsc-user mail archive on this topic but still not find a good > description on it.? I encounter similar errors in the practice as some > previous question-raisers did. > > Here is my simple code implementing the PCMG based on my understanding from > the user manual and tutorial.? It is supposed to have multigrid of only two > levels (finest and coarsest grid).? The error message indicates SEGV > segmentation violation because of this part of code.? Anything wrong with > this implementation? > > > ? DAGetMatrix(da, MATAIJ, &M); > > ? KSPCreate(PETSC_COMM_WORLD, &ksp); > ? KSPSetType(ksp, KSPGMRES); > ? KSPGetPC(ksp, &pcmg); > ? PCSetType(pcmg, PCMG); > > ? PCMGSetLevels(pcmg, 2, &PETSC_COMM_WORLD); > ? PCMGSetType(pcmg, PC_MG_MULTIPLICATIVE); > ? PCMGSetCycleType(pcmg, PC_MG_CYCLE_W); > ? PCMGSetCyclesOnLevel(pcmg, 0, 1); > ? PCMGSetCyclesOnLevel(pcmg, 1, 1); > > ? PCMGGetCoarseSolve(pcmg, &ksp); > > ? PCMGGetSmoother(pcmg, 0, &ksp); > ? PCMGGetSmoother(pcmg, 1, &ksp); > ? PCMGSetInterpolation(pcmg, 1, M); > ? PCMGSetRestriction(pcmg, 1, M); > > ? PCMGSetResidual(pcmg, 0, PCMGDefaultResidual, M); > ? PCMGSetResidual(pcmg, 1, PCMGDefaultResidual, M); > > > ?The mandatory remaining PCMGSetRhs( ), PCMGSetX( ) and PCMGSetR( ) > functions should be generated by Petsc automatically. > > It is tough to learn some Petsc functions when no detailed example and few > guidance is provided.? I wish you can add some tutorials codes on PCMG usage > in the future version of Petsc. > > > Thank you very much. > > > Zhisong Li > > > > From clemens.domanig at uibk.ac.at Wed May 18 02:15:27 2011 From: clemens.domanig at uibk.ac.at (Clemens Domanig) Date: Wed, 18 May 2011 09:15:27 +0200 Subject: [petsc-users] VecSetValue. fails with -O1/2/3 In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at> Message-ID: <4DD3720F.5090403@uibk.ac.at> First ValGrind didn't find anything but then the solution was found in Valgrind FAQ 4.6 (--tool=exp-ptrcheck) Thx for the help Jed Brown wrote: > Try with valgrind. The compiler is probably initializing a variable only > at -O0. > >> On May 16, 2011 8:28 AM, "Clemens Domanig" > > wrote: >> >> Hi everyone, >> >> maybe someone can explain to me what is going wrong with my code. >> >> In my programm I read values from a vector A and write them at new >> positions into a vector B. This works well as long as I don't use any >> optimization-flags when compiling the code. If I use -O1/2/3 the petsc >> tells me: "out of range error" at VecSetValue. >> It always happens at the some position although I'm doing this kind of >> operations at different places again. >> >> petsc-version 3.1.6 >> g++-version: gcc version 4.1.2 20080704 (Red Hat 4.1.2-48) >> >> Thx for your help - C. Domanig > From jed at 59A2.org Wed May 18 02:35:51 2011 From: jed at 59A2.org (Jed Brown) Date: Wed, 18 May 2011 09:35:51 +0200 Subject: [petsc-users] VecSetValue. fails with -O1/2/3 In-Reply-To: <4DD3720F.5090403@uibk.ac.at> References: <4DD0C3F7.5060504@uibk.ac.at> <4DD3720F.5090403@uibk.ac.at> Message-ID: On Wed, May 18, 2011 at 09:15, Clemens Domanig wrote: > First ValGrind didn't find anything but then the solution was found in > Valgrind FAQ 4.6 (--tool=exp-ptrcheck) Great, thanks for this tip. -------------- next part -------------- An HTML attachment was scrubbed... URL: From uerland at gmail.com Wed May 18 10:08:45 2011 From: uerland at gmail.com (Henning Sauerland) Date: Wed, 18 May 2011 17:08:45 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> Message-ID: <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> On 15.05.2011, at 15:59, Jed Brown wrote: > This is a problem, but you can still order unknowns in the vector so that velocity and pressure all come together, then AIJ/Inode will treat the components together which will tend to make ILU stronger. The results using the interlaced ordering are more pronounced than expected. The number of iterations did increase significantly using ILU(0) (~430 vs. ~170 per Picard iteration on 4 PEs) which makes it 4-5 times slower. However, the iteration count almost stays constant when moving to more processors! ILU(2) again reduces the iteration count (~100) while being slower compared to ILU(0). I really appreciate your suggestions so far! Henning From uerland at gmail.com Wed May 18 10:17:04 2011 From: uerland at gmail.com (Henning Sauerland) Date: Wed, 18 May 2011 17:17:04 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> Message-ID: <51D25D25-E135-4780-B909-AC8888DA9EA6@gmail.com> On 17.05.2011, at 04:54, Matthew Knepley wrote: > Anecdotally, I hear that XFEM can produce ill-conditioned matrices, where the conditioning arises from geometric factors. I don't think you can > a priori rule out differences in conditioning as the process count increases which come from changing geometry of the interface. > > Matt Anyway, the ill-conditoning should not be a problem here. Luckily, a diagonal scaling does a great job to improve the condition number. And as you said while the ill-conditioning usually arises from geometric factors I can identify the XFEM degrees of freedom which account for the ill-conditioning. Blocking such DOFs carefully also improves the condition number, which I usually not do as the diagonal scaling already leads to a condition number in the order of the standard FEM matrix. Henning From bsmith at mcs.anl.gov Wed May 18 12:46:34 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 18 May 2011 12:46:34 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> Message-ID: <6260C6F6-13F6-48BA-B1A6-F1FFE2C9FB46@mcs.anl.gov> On May 18, 2011, at 10:08 AM, Henning Sauerland wrote: > > On 15.05.2011, at 15:59, Jed Brown wrote: > >> This is a problem, but you can still order unknowns in the vector so that velocity and pressure all come together, then AIJ/Inode will treat the components together which will tend to make ILU stronger. > > The results using the interlaced ordering are more pronounced than expected. The number of iterations did increase significantly using ILU(0) (~430 vs. ~170 per Picard iteration on 4 PEs) which makes it 4-5 times slower. So interlacing the variables makes ILU() much worse in both iteration count and time? Barry > However, the iteration count almost stays constant when moving to more processors! ILU(2) again reduces the iteration count (~100) while being slower compared to ILU(0). > > > I really appreciate your suggestions so far! > > Henning From bsmith at mcs.anl.gov Wed May 18 13:41:54 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 18 May 2011 13:41:54 -0500 Subject: [petsc-users] V-cycle multigrid with matrix shells In-Reply-To: References: <0A20C9B7-C589-43A7-8B90-DD1AF7077EBF@mcs.anl.gov> <0007B63D-196F-4C19-B02E-DF279B3BBF32@mcs.anl.gov> <4FB1A2D2-1A4D-454C-9A72-A89795C209A2@mcs.anl.gov> Message-ID: This is happening because it is trying to setup the solver on a particular level but the matrix has not yet been provided to that solver with KSPSetOperators() 0]PETSC ERROR: Object is in wrong state! [0]PETSC ERROR: Matrix must be set first! ------------------------------------------------------------------------ [0]PETSC ERROR: PCSetUp() line 775 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: PCSetUp_MG() line 602 in src/ksp/pc/impls/mg/mg.c [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c In looking at your code it seems you are trying to calling KSPSetOperators(). For some reason in the logic of the code it is not getting to the line that needs to set the operators. I would run with the option -start_in_debugger noxterm and put a break in KSPSetFromOperators() and PCSetUp() then use "cont" and each time the debugger stops in KSPSetFromOperators() and PCSetUp() use "where" to see the current stack (where the code is). You can then match the KSPSetOperators() to each PCSetUp() and maybe figure out why the correct number of KSPSetOperators() are not being triggered in the code. I would do it myself except I don't have all the code so cannot build it and run it. Barry On May 17, 2011, at 7:39 PM, Sylvain Barbot wrote: > Barry, > > I have a call to KSPSetFromOptions() before KSPSetup() and KSPSolve() > already. Is there any way I can force the default to be SOR smoother > in the code instead of from the command line? With the command line > you suggested, I get a slightly different error: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Object is in wrong state! > [0]PETSC ERROR: Matrix must be set first! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 > 14:42:02 CDT 2010 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named > catalina.gps.caltech.edu by sbarbot Tue May 17 17:30:17 2011 > [0]PETSC ERROR: Libraries linked from > /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib > [0]PETSC ERROR: Configure run at Thu Oct 7 19:12:09 2010 > [0]PETSC ERROR: Configure options > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t > -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single > --with-petsc-arch=real4-with-debug > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: PCSetUp() line 775 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: PCSetUp_MG() line 602 in src/ksp/pc/impls/mg/mg.c > [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c > > there is no more reference to PCSetUp_BJacobi(), so it's a half > success. I attach the code setting up the solver/preconditioner for > reference (SUBROUTINE green_init_mg). > > btw, is there a way to better trace the location of the problem? i am > using the __FUNCT__ pre-compiling definition, for instance > > #undef __FUNCT__ > #define __FUNCT__ "matrixinterp" > SUBROUTINE matrixinterp(A,x,y,ierr) > > but it never shows up in the warning/error message. > > Thanks, > Sylvain > > > 2011/5/17 Barry Smith : >> >> This problem is because it is trying to use block Jacobi on a level as the smoother instead of SOR. You need to tell PCMG to use sor with -mg_levels_pc_type sor and make sure you have a KSPSetFromOptions() call before KSPSolve() so it catches those options. >> >> I'm sorry I forgot about the ourgetvecs() you did the correct fix. I will add that to petsc-dev also. >> >> >> Barry >> >> The default is block Jacobi with ILU on each process. >> >> >> On May 17, 2011, at 2:23 PM, Sylvain Barbot wrote: >> >>> Dear Barry, >>> >>> I made your suggested changes and recompiled the library. I also added >>> the function ourview and ourgetvecs from petsc-dev and changed the >>> last call of matshellsetoperations_ from >>> >>> PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL,"Cannot >>> set that matrix operation"); >>> >>> to >>> >>> PetscError(__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,0,1,"Cannot >>> set that matrix operation"); >>> >>> for compatibility with my current version of Petsc. >>> >>> I changed the matrix type of my smoothing matrix to MAT_SOR. Upon >>> running, i get the following error >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: No support for this operation for this object type! >>> [0]PETSC ERROR: This matrix type, shell, does not support getting >>> diagonal block! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 4, Fri Jul 30 >>> 14:42:02 CDT 2010 >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [0]PETSC ERROR: See docs/index.html for manual pages. >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: ./statici_petsc_1d_mg on a real4-wit named >>> catalina.gps.caltech.edu by sbarbot Tue May 17 12:08:10 2011 >>> [0]PETSC ERROR: Libraries linked from >>> /Users/sbarbot/Documents/work/petsc/real4-with-debug/lib >>> [0]PETSC ERROR: Configure run at Thu Oct 7 19:12:09 2010 >>> [0]PETSC ERROR: Configure options >>> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/084/Frameworks/mkl/lib/em64t >>> -with-mpi-dir=/shared/bin --with-debugging=1 --with-precision=single >>> --with-petsc-arch=real4-with-debug >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: PCSetUp_BJacobi() line 162 in src/ksp/pc/impls/bjacobi/bjacobi.c >>> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: PCSetUp_MG() line 574 in src/ksp/pc/impls/mg/mg.c >>> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >>> >>> btw, when I call MatShellSetOperation, what should be the type of the >>> shell matrix that performs the residuals, MAT_SOR as well, MAT_MULT or >>> something else? >>> >>> thanks again for your help. >>> Sylvain >>> >>> 2011/5/16 Barry Smith : >>>> >>>> On May 16, 2011, at 9:52 PM, Sylvain Barbot wrote: >>>> >>>>> Hi Barry, >>>>> >>>>> Thanks for your comments. >>>>> >>>>>>> CALL MatShellSetOperation(c%sA(l+1),MATOP_MULT,matrixsmoothfv,ierr); >>>>>> >>>>>> You say this is a smoother but you register it as a MATOP_MULT? What is the calling sequence of matrixsmoothfv() and what does it do? Smooth or multiply? MATOP_MULT is for registering a multiple MATOP_SOR (or MATOP_RELAX in older versions of PETSc) is for providing a smoother. >>>>> >>>>> My function matrixsmoothfv indeed performs the smoothing operation so >>>>> it not a multiplication. I can see no documentation about MATOP_SOR, >>>>> is there an example somewhere online? >>>> >>>> For each possible Matrix function there is the name, for matrix vector multiply it is MATOP_MULT and for smoothing it is MATOP_SOR and the calling sequence you use in your function that does the operation is the same as the matrix operation. For example for MatMult() it is Mat,Vec,Vec,PetscErrorCode (see below for the smoother). >>>>> Assigning MATOP_SOR to my >>>>> smoothing matrix now generates the following runtime error message: >>>>> >>>>> [0]PETSC ERROR: MatShellSetOperation_Fortran() line 107 in >>>>> src/mat/impls/shell/ftn-custom/zshellf.c >>>>> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 >>>>> rank 0 in job 37 catalina.gps.caltech.edu_50093 caused collective >>>>> abort of all ranks >>>>> exit status of rank 0: return code 1 >>>> >>>> This is because we hadn't added support for providing the MATOP_SOR for the Fortran interface. Edit src/mat/imples/shell/ftn-custom/zshellf.c and locate the function matshellsetoperation_() and put instead of it the following code. >>>> >>>> >>>> static PetscErrorCode oursor(Mat mat,Vec b,PetscReal omega,MatSORType flg,PetscReal shift,PetscInt its,PetscInt lits,Vec x) >>>> { >>>> PetscErrorCode ierr = 0; >>>> (*(PetscErrorCode (PETSC_STDCALL *)(Mat*,Vec*,PetscReal*,MatSORType*,PetscReal*,PetscInt*,PetscInt*,Vec*,PetscErrorCode*))(((PetscObject)mat)->fortran_func_pointers[9]))(&mat,&b,&omega,&flg,&shift,&its,&lits,&x,&ierr); >>>> return ierr; >>>> } >>>> >>>> void PETSC_STDCALL matshellsetoperation_(Mat *mat,MatOperation *op,PetscErrorCode (PETSC_STDCALL *f)(Mat*,Vec*,Vec*,PetscErrorCode*),PetscErrorCode *ierr) >>>> { >>>> MPI_Comm comm; >>>> >>>> *ierr = PetscObjectGetComm((PetscObject)*mat,&comm);if (*ierr) return; >>>> PetscObjectAllocateFortranPointers(*mat,10); >>>> if (*op == MATOP_MULT) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmult); >>>> ((PetscObject)*mat)->fortran_func_pointers[0] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_MULT_TRANSPOSE) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttranspose); >>>> ((PetscObject)*mat)->fortran_func_pointers[2] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_MULT_ADD) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmultadd); >>>> ((PetscObject)*mat)->fortran_func_pointers[1] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_MULT_TRANSPOSE_ADD) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourmulttransposeadd); >>>> ((PetscObject)*mat)->fortran_func_pointers[3] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_GET_DIAGONAL) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetdiagonal); >>>> ((PetscObject)*mat)->fortran_func_pointers[4] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_DIAGONAL_SCALE) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalscale); >>>> ((PetscObject)*mat)->fortran_func_pointers[5] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_DIAGONAL_SET) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourdiagonalset); >>>> ((PetscObject)*mat)->fortran_func_pointers[6] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_GET_VECS) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourgetvecs); >>>> ((PetscObject)*mat)->fortran_func_pointers[7] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_VIEW) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)ourview); >>>> ((PetscObject)*mat)->fortran_func_pointers[8] = (PetscVoidFunction)f; >>>> } else if (*op == MATOP_SOR) { >>>> *ierr = MatShellSetOperation(*mat,*op,(PetscVoidFunction)oursor); >>>> ((PetscObject)*mat)->fortran_func_pointers[9] = (PetscVoidFunction)f; >>>> } else { >>>> PetscError(comm,__LINE__,"MatShellSetOperation_Fortran",__FILE__,__SDIR__,PETSC_ERR_ARG_WRONG,PETSC_ERROR_INITIAL, >>>> "Cannot set that matrix operation"); >>>> *ierr = 1; >>>> } >>>> } >>>> >>>> >>>> Then in that directory run make to update the PETSc libraries. >>>> >>>> Now note the calling sequence of the relaxation function you need to provide. It is PetscErrorCode MatSOR(Mat mat,Vec b,PetscReal omega,MatSORType flag,PetscReal shift,PetscInt its,PetscInt lits,Vec x,PetscErrorCode ierr) >>>> in your function you can ignore the omega, the flag, and the shift argument. You function should do its*lits smoothing steps. >>>> >>>> Once you have this going please let us know what happens. >>>> >>>> Barry >>>> >>>> >>>> >>>> >>>>> >>>>>>> I also provide work space... >>>>>>> Despite that, I have a crash upon running KSPSetup() with "[0]PETSC >>>>>>> ERROR: PCSetUp_BJacobi()". >>>>>> >>>>>> It is trying to use bjacobi because that is the default solver, you need to tell it to use some other solver explicitly if you do not want it to use block Jacobi. So if you registered your matrixsmoothfv() as a MATOP_SOR then you can use -mg_levels_pc_type sor to have it use that smoother. What solver options are you running with? >>>>> >>>>> I setup the solver with: >>>>> >>>>> CALL PCMGSetType(c%pc,PC_MG_MULTIPLICATIVE,ierr); >>>>> >>>>> but this error might not be representative of the problem. upstream, >>>>> my smoothing matrices were not set up properly. >>>>> >>>>>>> MatGetVecs() line 7265 in src/mat/interface/matrix.c >>>>>>> [0]PETSC ERROR: KSPGetVecs() line 806 in src/ksp/ksp/interface/iterativ.c >>>>>>> [0]PETSC ERROR: KSPSetUp_GMRES() line 94 in src/ksp/ksp/impls/gmres/gmres.c >>>>>>> [0]PETSC ERROR: KSPSetUp() line >>>>>>> >>>>>> This is not an indication that it is trying to use a direct solver, this is an indication then you never provided ANY kind of matrix at this level. You need to provide a matrix operator on each level for it to get work vectors from etc. It would be help to have the COMPLETE error message here so I can see exactly what is happening instead of a fragment making me guess what the problem is. >>>> >>>> >> >> > From bartlomiej.wach at yahoo.pl Wed May 18 15:54:06 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Wed, 18 May 2011 21:54:06 +0100 (BST) Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: Message-ID: <250859.33295.qm@web28309.mail.ukl.yahoo.com> Hello again, I am wondering if it is possibile to have a matrix be stored across several machines but have only one process to set its values? Can I just create the matrix and then simply setownership range for process 0 to full size of the matrix and zero for others? Thank You for an answer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed May 18 16:05:59 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 18 May 2011 16:05:59 -0500 Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: <250859.33295.qm@web28309.mail.ukl.yahoo.com> References: <250859.33295.qm@web28309.mail.ukl.yahoo.com> Message-ID: On Wed, May 18, 2011 at 3:54 PM, Bart?omiej W wrote: > Hello again, > > I am wondering if it is possibile to have a matrix be stored across several > machines but have only one process to set its values? Can I just create the > matrix and then simply setownership range for process 0 to full size of the > matrix and zero for others? > You can create a parallel matrix, and then only set values from 1 process. Its not efficient, but it will work. Matt > Thank You for an answer. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed May 18 18:32:42 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 18 May 2011 18:32:42 -0500 Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: References: <250859.33295.qm@web28309.mail.ukl.yahoo.com> Message-ID: <7C0E4613-0FCF-453C-B307-4F244A0903CE@mcs.anl.gov> On May 18, 2011, at 4:05 PM, Matthew Knepley wrote: > On Wed, May 18, 2011 at 3:54 PM, Bart?omiej W wrote: > Hello again, > > I am wondering if it is possibile to have a matrix be stored across several machines but have only one process to set its values? Can I just create the matrix and then simply setownership range for process 0 to full size of the matrix and zero for others? > > You can create a parallel matrix, and then only set values from 1 process. Its not efficient, but it will work. For big problems it will be terribly slow, so slow as to be idiotic. Note in our users manual introduction we specifically say PETSc should not be used to attempt to provide a ?parallel linear solver? in an otherwise sequential code. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be to expect any kind of reasonable performance. Do not expect to generate your matrix sequentially and then ?use PETSc? to solve the linear system in parallel. > > Matt > > Thank You for an answer. > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener From rongtian at ncic.ac.cn Thu May 19 01:43:08 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Thu, 19 May 2011 14:43:08 +0800 Subject: [petsc-users] SNES never called user FormJacobian in debugging mode In-Reply-To: References: <4DD0C3F7.5060504@uibk.ac.at><4DD3720F.5090403@uibk.ac.at> Message-ID: <18597D42B5424DE28FD8020177841CC6@rti458laptop> I encountered a strange problem. I am using VC2008. When I debuged the code line by line using the VC, the user FormJacobian was never called, instead, the SNES seems use matrix free Jacobian. But if I run the code in command line (no debugger attached), the log file showed that the user FormJacobian was called. What is the reason for this? -Rong -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Thu May 19 02:54:22 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 19 May 2011 08:54:22 +0100 (BST) Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: <7C0E4613-0FCF-453C-B307-4F244A0903CE@mcs.anl.gov> Message-ID: <594911.42531.qm@web28310.mail.ukl.yahoo.com> Thank You for Your answers. I have another question I wanted to try a 64 bit system out and got the ubuntu with 2 GB RAM and additional 11 GB swap and still I get memory allocation error like this: [0] Maximum memory PetscMalloc()ed 2,514,564,304 OS cannot compute size of entire process Do I miss something when I compile the binaries? Regards --- 18.5.11 (?r), Barry Smith napisa?(a): Od: Barry Smith Temat: Re: [petsc-users] Matrix stored on many machines, filled by one process Do: "PETSc users list" Data: 18 Maj 2011 (?roda), 23:32 On May 18, 2011, at 4:05 PM, Matthew Knepley wrote: > On Wed, May 18, 2011 at 3:54 PM, Bart?omiej W wrote: > Hello again, > > I am wondering if it is possibile to have a matrix be stored across several machines but have only one process to set its values? Can I just create the matrix and then simply setownership range for process 0 to full size of the matrix and zero for others? > > You can create a parallel matrix, and then only set values from 1 process. Its not efficient, but it will work. ???For big problems it will be terribly slow, so slow as to be idiotic. ? ? Note in our users manual introduction we specifically say PETSc should not be used to attempt to provide a ?parallel linear solver? in an otherwise sequential code. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be to expect any kind of reasonable performance. Do not expect to generate your matrix sequentially and then ?use PETSc? to solve the linear system in parallel. > >? ? Matt >? > Thank You for an answer. > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 19 03:06:04 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 19 May 2011 10:06:04 +0200 Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: <594911.42531.qm@web28310.mail.ukl.yahoo.com> References: <7C0E4613-0FCF-453C-B307-4F244A0903CE@mcs.anl.gov> <594911.42531.qm@web28310.mail.ukl.yahoo.com> Message-ID: On Thu, May 19, 2011 at 09:54, Bart?omiej W wrote: > I wanted to try a 64 bit system out and got the ubuntu with 2 GB RAM and > additional 11 GB swap and still I get memory allocation error like this: > > [0] Maximum memory PetscMalloc()ed 2,514,564,304 OS cannot compute size of > entire process This is in PetscMallocComputeLog() and is not really an error. It's really intended to be called when your program completes since the "maximum usage" is calculated when objects are destroyed instead of when they are allocated. (This could be handled better.) What is your code doing and what options are you running with? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Thu May 19 04:09:05 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 19 May 2011 10:09:05 +0100 (BST) Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: Message-ID: <909595.5412.qm@web28312.mail.ukl.yahoo.com> The error is like: [0]Total space allocated 2514,564,304 bytes (...) [0]PETSC ERROR: Memory allocated -2147483648 Memory used by process 1848942592 [0]PETSC ERROR: Memory requested 1578,397,300! Caused by the code ? ierr = MatCreate(PETSC_COMM_WORLD,&Dinv);CHKERRQ(ierr); ? ierr = MatSetSizes(Dinv,PETSC_DECIDE,PETSC_DECIDE,n,n);CHKERRQ(ierr); ? ierr = MatSetFromOptions(Dinv);CHKERRQ(ierr); ? ierr = MatAssemblyBegin(Dinv,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); where n = 39459925 Since it's second such matrix of this size in the code (first one gets enough memory)? and by watching the system resources I see that there should be enough memory to allocate this one initially, I wonder what might be wrong. --- 19.5.11 (Czw), Jed Brown napisa?(a): Od: Jed Brown Temat: Re: [petsc-users] Matrix stored on many machines, filled by one process Do: "PETSc users list" Data: 19 Maj 2011 (Czwartek), 8:06 On Thu, May 19, 2011 at 09:54, Bart?omiej W wrote: I wanted to try a 64 bit system out and got the ubuntu with 2 GB RAM and additional 11 GB swap and still I get memory allocation error like this: [0] Maximum memory PetscMalloc()ed 2,514,564,304 OS cannot compute size of entire process This is in PetscMallocComputeLog() and is not really an error. It's really intended to be called when your program completes since the "maximum usage" is calculated when objects are destroyed instead of when they are allocated. (This could be handled better.) What is your code doing and what options are you running with? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Thu May 19 04:21:38 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 19 May 2011 10:21:38 +0100 (BST) Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: Message-ID: <881037.19724.qm@web28308.mail.ukl.yahoo.com> --- 19.5.11 (Czw), Jed Brown napisa?(a): Od: Jed Brown Temat: Re: [petsc-users] Matrix stored on many machines, filled by one process Do: "PETSc users list" Data: 19 Maj 2011 (Czwartek), 8:06 On Thu, May 19, 2011 at 09:54, Bart?omiej W wrote: I wanted to try a 64 bit system out and got the ubuntu with 2 GB RAM and additional 11 GB swap and still I get memory allocation error like this: [0] Maximum memory PetscMalloc()ed 2,514,564,304 OS cannot compute size of entire process This is in PetscMallocComputeLog() and is not really an error. It's really intended to be called when your program completes since the "maximum usage" is calculated when objects are destroyed instead of when they are allocated. (This could be handled better.) What is your code doing and what options are you running with? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 19 04:28:56 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 19 May 2011 11:28:56 +0200 Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: <909595.5412.qm@web28312.mail.ukl.yahoo.com> References: <909595.5412.qm@web28312.mail.ukl.yahoo.com> Message-ID: On Thu, May 19, 2011 at 11:09, Bart?omiej W wrote: > [0]Total space allocated 2514,564,304 bytes > (...) > [0]PETSC ERROR: Memory allocated -2147483648 Memory used by process > 1848942592 > [0]PETSC ERROR: Memory requested 1578,397,300! Please always send the whole error trace. I have updated petsc-dev so the sizes are not incorrectly truncated like you see here. Are you sure you have built a 64-bit executable (run "file your-executable")? > > > Caused by the code > > ierr = MatCreate(PETSC_COMM_WORLD,&Dinv);CHKERRQ(ierr); > ierr = MatSetSizes(Dinv,PETSC_DECIDE,PETSC_DECIDE,n,n);CHKERRQ(ierr); > ierr = MatSetFromOptions(Dinv);CHKERRQ(ierr); > ierr = MatAssemblyBegin(Dinv,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); Is this in parallel? How many processors? Does the name "Dinv" imply that this matrix is supposed to be the inverse of something? Why are you assembling without putting anything in the matrix? > > where n = 39459925 How many nonzeros per row? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Thu May 19 05:00:12 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 19 May 2011 11:00:12 +0100 (BST) Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: Message-ID: <898959.18727.qm@web28306.mail.ukl.yahoo.com> On Thu, May 19, 2011 at 11:09, Bart?omiej W wrote: [0]Total space allocated 2514,564,304 bytes (...) [0]PETSC ERROR: Memory allocated -2147483648 Memory used by process 1848942592 [0]PETSC ERROR: Memory requested 1578,397,300! Please always send the whole error trace. I have updated petsc-dev so the sizes are not incorrectly truncated like you see here. Are you sure you have built a 64-bit executable (run "file your-executable")? ?This might be the problem, it occurs it is 32 bit, tried to compile with -m64 though? but got /usr/bin/ld: skipping incompatible .../petsc-3.1-p8/linux-gnu-c-debug/lib/libpetsc.a when searching for -lpetsc Caused by the code ? ierr = MatCreate(PETSC_COMM_WORLD,&Dinv);CHKERRQ(ierr); ? ierr = MatSetSizes(Dinv,PETSC_DECIDE,PETSC_DECIDE,n,n);CHKERRQ(ierr); ? ierr = MatSetFromOptions(Dinv);CHKERRQ(ierr); ? ierr = MatAssemblyBegin(Dinv,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr); Is this in parallel? How many processors? Does the name "Dinv" imply that this matrix is supposed to be the inverse of something? Why are you assembling without putting anything in the matrix? where n = 39459925 How many nonzeros per row? I try to run in on single processor right now. Yes, Dinv is supposed to be an inverse of a diagonal matrix (1 nnz/row) Made it sparse and the app went on but stopped a bit further on next allocation, problem is the 32 bit build while it should be 64 but I cant get it right. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 19 05:04:01 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 19 May 2011 12:04:01 +0200 Subject: [petsc-users] Matrix stored on many machines, filled by one process In-Reply-To: <898959.18727.qm@web28306.mail.ukl.yahoo.com> References: <898959.18727.qm@web28306.mail.ukl.yahoo.com> Message-ID: On Thu, May 19, 2011 at 12:00, Bart?omiej W wrote: > This might be the problem, it occurs it is 32 bit, tried to compile with > -m64 though but got > > /usr/bin/ld: skipping incompatible > .../petsc-3.1-p8/linux-gnu-c-debug/lib/libpetsc.a when searching for -lpetsc > You need to reconfigure PETSc using 64-bit compilers, then rebuild the library. For a diagonal matrix, you can call MatSeqAIJSetPreallocation(Dinv,1,PETSC_NULL) immediately after MatSetFromOptions() and it will use much less memory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu May 19 07:42:17 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 19 May 2011 07:42:17 -0500 Subject: [petsc-users] SNES never called user FormJacobian in debugging mode In-Reply-To: <18597D42B5424DE28FD8020177841CC6@rti458laptop> References: <4DD0C3F7.5060504@uibk.ac.at><4DD3720F.5090403@uibk.ac.at> <18597D42B5424DE28FD8020177841CC6@rti458laptop> Message-ID: <465C974E-AA0F-4D8C-8B25-89B7FE84F9F4@mcs.anl.gov> On May 19, 2011, at 1:43 AM, Tian(ICT) wrote: > I encountered a strange problem. > I am using VC2008. When I debuged the code line by line using the VC, the user FormJacobian was never called, instead, the SNES seems use matrix free Jacobian. But if I run the code in command line (no debugger attached), the log file showed that the user FormJacobian was called. What is the reason for this? There is not logical explanation for this. Running in the debugger should not trigger a different code path then not in the debugger. Also SNES will never use matrix-free unless you specifically ask for it. What can happen is that SNES may decide (for some reason) that the initial residual norm is small enough and hence never try a Newton step. If there are some uninitialized variables or other problems in the code the run in and out of the debugger could be different hence different "function values". Run with -snes_monitor -ksp_monitor_true_residual -snes_view -snes_converged_reason -ksp_converged_reason. Ideally in this situation you would run with valgrind http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind or a similar tool to check that the code is memory corruption free. Barry > > -Rong From Juergen.Duerrwang at iosb.fraunhofer.de Thu May 19 09:18:39 2011 From: Juergen.Duerrwang at iosb.fraunhofer.de (=?iso-8859-1?Q?D=FCrrwang=2C_J=FCrgen?=) Date: Thu, 19 May 2011 16:18:39 +0200 Subject: [petsc-users] How to get a duoble array into petsc Vec? Message-ID: Hi, I am trying to put data form a double c array to an petsc vector. I tried to loop over each element like this petsc Vec[i] = double array[i], but I need a cast for that....has anybody an idea? Thanks J?rgen -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mathieu.Morlighem at jpl.nasa.gov Thu May 19 09:50:31 2011 From: Mathieu.Morlighem at jpl.nasa.gov (Morlighem, Mathieu (334H-Affiliate)) Date: Thu, 19 May 2011 07:50:31 -0700 Subject: [petsc-users] How to get a duoble array into petsc Vec? In-Reply-To: References: Message-ID: Try this instead: for(i=0;i From knepley at gmail.com Thu May 19 10:46:52 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 May 2011 10:46:52 -0500 Subject: [petsc-users] How to get a duoble array into petsc Vec? In-Reply-To: References: Message-ID: 2011/5/19 D?rrwang, J?rgen > Hi, > > > > I am trying to put data form a double c array to an petsc vector. I tried > to loop over each element like this petsc Vec[i] = double array[i], but I > need a cast for that?.has anybody an idea? > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/vec/vec/examples/tutorials/ex3.c.html Matt > > > Thanks J?rgen > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Thu May 19 11:51:41 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Thu, 19 May 2011 11:51:41 -0500 (CDT) Subject: [petsc-users] How to get a duoble array into petsc Vec? In-Reply-To: Message-ID: <1690850033.85263.1305823901380.JavaMail.root@zimbra.anl.gov> You could use VecCreateSeqWithArray() http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecCreateSeqWithArray.html Shri ----- Original Message ----- Hi, I am trying to put data form a double c array to an petsc vector. I tried to loop over each element like this petsc Vec[i] = double array[i], but I need a cast for that?.has anybody an idea? Thanks J?rgen -------------- next part -------------- An HTML attachment was scrubbed... URL: From lizs at mail.uc.edu Thu May 19 19:56:53 2011 From: lizs at mail.uc.edu (Li, Zhisong (lizs)) Date: Fri, 20 May 2011 00:56:53 +0000 Subject: [petsc-users] ML Optimization and local preconditioners Message-ID: Hi, Petsc Team, Recently I tested my 3D structured Poisson-style problem with ML and BoomerAMG preconditioner respectively. In comparison, ML is more efficient in preconditioning and RAM usage, but it requires 2 times more iterations on the same KSP solver, bringing down the overall efficiency. And both PCs don't scale well. I wonder if there's any specific approach to optimizing ML to reduce KSP iterations by setting certain command line options. I also saw in some previous petsc mail archives mentioning the "local preconditioner". As some important PC like PCILU and PCICC are not available for parallel processing, it may be beneficial to apply them as local preconditioners. The question is how to setup a local preconditioner? Thank you veru much. Zhisong Li -------------- next part -------------- An HTML attachment was scrubbed... URL: From austin at txcorp.com Thu May 19 21:58:58 2011 From: austin at txcorp.com (Travis Austin) Date: Thu, 19 May 2011 19:58:58 -0700 Subject: [petsc-users] ML Optimization and local preconditioners In-Reply-To: References: Message-ID: <3B80E68D-8889-45C7-A31A-4790ECCB4DEC@txcorp.com> Hi, Not on the PETSc team but some experience with these two multilevel preconditioners. For starters take a look at this publication by one of the HYPRE team members on parameter choices for 2D and 3D Poisson problems that deliver the best performance. Pay particular attention to p. 18-22. There are many knobs with these solvers (in particular BoomerAMG) and they may need tweaking to improve performance. https://computation.llnl.gov/casc/linear_solvers/pubs/yang1.pdf Also, what is your definition of poor scalability? With respect to increasing processor count (i.e., parallel scalability) or with respect to performance based on increasing problem size? Both of these preconditiioners have been thoroughly tested for Poisson-style problems and I'd be surprised if you don't get (at least) good scalability with respect to problem size? Travis ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Travis Austin, Ph.D. Tech-X Corporation 5621 Arapahoe Ave, Suite A Boulder, CO 80303 austin at txcorp.com ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On May 19, 2011, at 5:56 PM, Li, Zhisong (lizs) wrote: > Hi, Petsc Team, > > Recently I tested my 3D structured Poisson-style problem with ML and BoomerAMG preconditioner respectively. In comparison, ML is more efficient in preconditioning and RAM usage, but it requires 2 times more iterations on the same KSP solver, bringing down the overall efficiency. And both PCs don't scale well. I wonder if there's any specific approach to optimizing ML to reduce KSP iterations by setting certain command line options. > > I also saw in some previous petsc mail archives mentioning the "local preconditioner". As some important PC like PCILU and PCICC are not available for parallel processing, it may be beneficial to apply them as local preconditioners. The question is how to setup a local preconditioner? > > > Thank you veru much. > > > > Zhisong Li -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.mayhem23 at gmail.com Fri May 20 01:10:02 2011 From: dave.mayhem23 at gmail.com (Dave May) Date: Fri, 20 May 2011 08:10:02 +0200 Subject: [petsc-users] ML Optimization and local preconditioners In-Reply-To: References: Message-ID: One of nice aspects of using ML compared to BoomerAMG is you have much more control over how you configure the smoother on each level. ILU and ICC are not directly available in parallel, but you can use a block variant of each, which applies ILU/ICC on the diagonal block of the matrix which is local to each processor. To configure this, use these options -mg_levels_pc_type bjacobi -mg_levels_sub_pc_type ilu or -mg_levels_pc_type bjacobi -mg_levels_sub_pc_type icc Dave On 20 May 2011 02:56, Li, Zhisong (lizs) wrote: > Hi, Petsc Team, > > Recently I tested my 3D structured Poisson-style problem with ML and > BoomerAMG preconditioner respectively.? In comparison, ML is more efficient > in preconditioning and RAM usage, but it requires 2 times more iterations on > the same KSP solver, bringing down the overall efficiency.? And both PCs > don't scale well.? I wonder if there's any specific approach to optimizing > ML to reduce KSP iterations by setting certain command line options. > > I also saw in some previous petsc mail archives mentioning the "local > preconditioner".? As some important PC like PCILU and PCICC are not > available for parallel processing, it may be beneficial to apply them as > local preconditioners.? The question is how to setup a local preconditioner? > > > Thank you veru much. > > > > Zhisong Li > From uerland at gmail.com Fri May 20 01:31:06 2011 From: uerland at gmail.com (Henning Sauerland) Date: Fri, 20 May 2011 08:31:06 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <6260C6F6-13F6-48BA-B1A6-F1FFE2C9FB46@mcs.anl.gov> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> <6260C6F6-13F6-48BA-B1A6-F1FFE2C9FB46@mcs.anl.gov> Message-ID: Am 18.05.2011 um 19:46 schrieb Barry Smith: > > So interlacing the variables makes ILU() much worse in both iteration count and time? Yes. I guess the reason is that the inode routines are not used with the interlaced ordering: interlaced: KSP Object: type: lgmres GMRES: restart=200, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 LGMRES: aug. dimension=2 LGMRES: number of matvecs=592 maximum iterations=10000, initial guess is zero tolerances: relative=1e-08, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: type: asm Additive Schwarz: total subdomain blocks = 16, amount of overlap = 1 Additive Schwarz: restriction/interpolation type - RESTRICT Local solve is same for all blocks, in the following KSP and PC objects: KSP Object:(sub_) type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object:(sub_) type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 1e-12 using diagonal shift to prevent zero pivot matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Matrix Object: type=seqaij, rows=144120, cols=144120 package used to perform factorization: petsc total: nonzeros=14238265, allocated nonzeros=14238265 not using I-node routines linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=144120, cols=144120 total: nonzeros=14238265, allocated nonzeros=14238265 not using I-node routines linear system matrix = precond matrix: Matrix Object: type=mpiaij, rows=548908, cols=548908 total: nonzeros=55971327, allocated nonzeros=93314360 not using I-node (on process 0) routines non-interlaced: KSP Object: type: lgmres GMRES: restart=200, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 LGMRES: aug. dimension=2 LGMRES: number of matvecs=334 maximum iterations=10000, initial guess is zero tolerances: relative=1e-08, absolute=1e-50, divergence=10000 left preconditioning diagonally scaled system using PRECONDITIONED norm type for convergence test PC Object: type: asm Additive Schwarz: total subdomain blocks = 16, amount of overlap = 1 Additive Schwarz: restriction/interpolation type - RESTRICT Local solve is same for all blocks, in the following KSP and PC objects: KSP Object:(sub_) type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object:(sub_) type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 1e-12 using diagonal shift to prevent zero pivot matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Matrix Object: type=seqaij, rows=41200, cols=41200 package used to perform factorization: petsc total: nonzeros=3651205, allocated nonzeros=3651205 using I-node routines: found 15123 nodes, limit used is 5 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=41200, cols=41200 total: nonzeros=3651205, allocated nonzeros=3651205 using I-node routines: found 15123 nodes, limit used is 5 linear system matrix = precond matrix: Matrix Object: type=mpiaij, rows=548908, cols=548908 total: nonzeros=55971327, allocated nonzeros=112526140 using I-node (on process 0) routines: found 13156 nodes, limit used is 5 From Juergen.Duerrwang at iosb.fraunhofer.de Fri May 20 03:13:09 2011 From: Juergen.Duerrwang at iosb.fraunhofer.de (=?utf-8?B?RMO8cnJ3YW5nLCBKw7xyZ2Vu?=) Date: Fri, 20 May 2011 10:13:09 +0200 Subject: [petsc-users] How to get a duoble array into petsc Vec? In-Reply-To: <1690850033.85263.1305823901380.JavaMail.root@zimbra.anl.gov> References: <1690850033.85263.1305823901380.JavaMail.root@zimbra.anl.gov> Message-ID: Thanks for your reply, it works fine! J?rgen You could use VecCreateSeqWithArray() http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecCreateSeqWithArray.html Shri ________________________________ Hi, I am trying to put data form a double c array to an petsc vector. I tried to loop over each element like this petsc Vec[i] = double array[i], but I need a cast for that?.has anybody an idea? Thanks J?rgen -------------- next part -------------- An HTML attachment was scrubbed... URL: From Juergen.Duerrwang at iosb.fraunhofer.de Fri May 20 06:23:09 2011 From: Juergen.Duerrwang at iosb.fraunhofer.de (=?iso-8859-1?Q?D=FCrrwang=2C_J=FCrgen?=) Date: Fri, 20 May 2011 13:23:09 +0200 Subject: [petsc-users] creating csr matrix from 3 arrays Message-ID: Hi I am trying to generate a matrix(nxn) from 3 arrays(1D) which I have read from 3 single textfiles: PetscInt pcol[] (dimension of nz): points to the integer array of length nz that holds the column indices of the corresponding elements in pval PetscInt prow[] (dimesion of n) PetscScalar pval[] (dimension of nz) I tried this: MatSetValues(A,n,prow,nz,pcol,pval,INSERT_VALUES); MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); Can I insert the hole data into matrix A or should I loop over it? Thanks J?rgen -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Fri May 20 06:33:00 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 20 May 2011 13:33:00 +0200 Subject: [petsc-users] creating csr matrix from 3 arrays In-Reply-To: References: Message-ID: 2011/5/20 D?rrwang, J?rgen > MatSetValues(A,n,prow,nz,pcol,pval,INSERT_VALUES); This function inserts logically dense blocks. You would loop over rows: for (row=0; row From Juergen.Duerrwang at iosb.fraunhofer.de Fri May 20 07:45:26 2011 From: Juergen.Duerrwang at iosb.fraunhofer.de (=?utf-8?B?RMO8cnJ3YW5nLCBKw7xyZ2Vu?=) Date: Fri, 20 May 2011 14:45:26 +0200 Subject: [petsc-users] creating csr matrix from 3 arrays In-Reply-To: References: Message-ID: Thanks a lot, without you I would sit for my code until midnight Von: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov] Im Auftrag von Jed Brown Gesendet: Freitag, 20. Mai 2011 13:33 An: PETSc users list Betreff: Re: [petsc-users] creating csr matrix from 3 arrays 2011/5/20 D?rrwang, J?rgen > MatSetValues(A,n,prow,nz,pcol,pval,INSERT_VALUES); This function inserts logically dense blocks. You would loop over rows: for (row=0; row From thomas.witkowski at tu-dresden.de Fri May 20 07:55:19 2011 From: thomas.witkowski at tu-dresden.de (Thomas Witkowski) Date: Fri, 20 May 2011 14:55:19 +0200 Subject: [petsc-users] Direct solver for 3D-FEM Message-ID: <4DD664B7.6030308@tu-dresden.de> Could one of you explain me why direct solvers (I make use of UMFPACK) seems to work quite bad for 3D-FEM? For a small test case, I take a unite square/box and refine it globally (bisectioning of triangle/tetrahedrons). I solve a six order PDE (that leads to symmetric but indefinite matrices) on it. In 2D the resulting system has 49.923 rows with 808.199 non zeros, UMFPACK solves the system within 2.8 seconds. For 3D I've refine the box such that the resulting matrix has more or less the same number of non zeros, in this case 898.807 on 27.027 rows. UMFPACK needs 32 seconds to solve it, so more than 10 time as for the 2D case. Even if I make use of fourth order Lagrange functions for the 2D case, which leads to much denser matrices (49.923 rows and 2.705.927 non zeros), UMFPACK solves it within 3.2 seconds. Is there any good reason for it? Thomas From jed at 59A2.org Fri May 20 08:05:26 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 20 May 2011 15:05:26 +0200 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: <4DD664B7.6030308@tu-dresden.de> References: <4DD664B7.6030308@tu-dresden.de> Message-ID: On Fri, May 20, 2011 at 14:55, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > Could one of you explain me why direct solvers (I make use of UMFPACK) > seems to work quite bad for 3D-FEM? For a small test case, I take a unite > square/box and refine it globally (bisectioning of triangle/tetrahedrons). I > solve a six order PDE (that leads to symmetric but indefinite matrices) on > it. In 2D the resulting system has 49.923 rows with 808.199 non zeros, > UMFPACK solves the system within 2.8 seconds. For 3D I've refine the box > such that the resulting matrix has more or less the same number of non > zeros, in this case 898.807 on 27.027 rows. UMFPACK needs 32 seconds to > solve it, so more than 10 time as for the 2D case. Even if I make use of > fourth order Lagrange functions for the 2D case, which leads to much denser > matrices (49.923 rows and 2.705.927 non zeros), UMFPACK solves it within 3.2 > seconds. Is there any good reason for it? It has to do with maximal independent sets, or (more roughly) that the ratio of surface area/volume (3D) is larger than perimeter/area (2D). This also affects the amount of communication. You can try using MUMPS instead of Umfpack, but it won't change the asymptotics. For PDE problems in 2D, direct solvers with optimal ordering need O(n log n) memory and O(n^{1.5}) flops. In 3D, they need O(n^{4/3}) memory and O(n^2) flops. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.witkowski at tu-dresden.de Fri May 20 08:16:21 2011 From: thomas.witkowski at tu-dresden.de (Thomas Witkowski) Date: Fri, 20 May 2011 15:16:21 +0200 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: References: <4DD664B7.6030308@tu-dresden.de> Message-ID: <4DD669A5.3080606@tu-dresden.de> Jed Brown wrote: > On Fri, May 20, 2011 at 14:55, Thomas Witkowski > > wrote: > > Could one of you explain me why direct solvers (I make use of > UMFPACK) seems to work quite bad for 3D-FEM? For a small test > case, I take a unite square/box and refine it globally > (bisectioning of triangle/tetrahedrons). I solve a six order PDE > (that leads to symmetric but indefinite matrices) on it. In 2D > the resulting system has 49.923 rows with 808.199 non zeros, > UMFPACK solves the system within 2.8 seconds. For 3D I've refine > the box such that the resulting matrix has more or less the same > number of non zeros, in this case 898.807 on 27.027 rows. UMFPACK > needs 32 seconds to solve it, so more than 10 time as for the 2D > case. Even if I make use of fourth order Lagrange functions for > the 2D case, which leads to much denser matrices (49.923 rows and > 2.705.927 non zeros), UMFPACK solves it within 3.2 seconds. Is > there any good reason for it? > > > It has to do with maximal independent sets, or (more roughly) that the > ratio of surface area/volume (3D) is larger than perimeter/area (2D). > This also affects the amount of communication. You can try using MUMPS > instead of Umfpack, but it won't change the asymptotics. > > For PDE problems in 2D, direct solvers with optimal ordering need O(n > log n) memory and O(n^{1.5}) flops. In 3D, they need O(n^{4/3}) memory > and O(n^2) flops. Thanks for the information. One related question: The system that is solved here is a 3x3 block matrix: Two of these blocks are empty, two are the discretized (FEM) identity, three are the discretized Laplace operator and one block is the discretization of a complex sum of operators. At all, there are only three different blocks. Can I make somehow advantages of this structure? I could create the three inverses of the three different blocks, but exists than a cheap/simple way to produce the action of the inverse of the whole 3x3 block matrix? Thomas From jed at 59A2.org Fri May 20 08:35:46 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 20 May 2011 15:35:46 +0200 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: <4DD669A5.3080606@tu-dresden.de> References: <4DD664B7.6030308@tu-dresden.de> <4DD669A5.3080606@tu-dresden.de> Message-ID: On Fri, May 20, 2011 at 15:16, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > Thanks for the information. One related question: The system that is solved > here is a 3x3 block matrix: Two of these blocks are empty, two are the > discretized (FEM) identity, three are the discretized Laplace operator What equation? Have you assumed special boundary conditions? Problems like elasticity and Stokes normally need a symmetric stress tensor. Only for constant coefficients and certain boundary conditions, is it okay to use the "block Laplacian" form. > and one block is the discretization of a complex sum of operators. At all, > there are only three different blocks. Can I make somehow advantages of this > structure? I could create the three inverses of the three different blocks, > but exists than a cheap/simple way to produce the action of the inverse of > the whole 3x3 block matrix? For the special case of a Kronecker product, you can solve using solves with the blocks, but not in the general case. Consider the structured grid 2D Laplacian [A, -I; -I, A] where A is the tridiagonal problem for a 1D Laplacian plus 2*I on the diagonal. If you could build a direct solver for the 2D using direct solvers for the 1D problem, then you would have an O(n) algorithm for a direct solve of a 2D Laplacian. You can, however, build an iterative method from the blocks. If each block is built separately, you can (with petsc-dev) put them into MatNest and use PCFieldSplit with direct solves in splits. An alternative is to interlace the degrees of freedom and use the (S)BAIJ matrix format. This will reduce the amount of memory needed for your matrix (by using a more efficient storage format) and will improve the memory performance. Unfortunately, there are no third-party direct solvers for BAIJ. Note that your assembly code can always call MatSetValuesBlocked() (or ..BlockedLocal()) regardless of the matrix format, so you can switch between AIJ and BAIJ at run-time. Why bother with a direct solver if your problem has constant coefficients? -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.witkowski at tu-dresden.de Fri May 20 08:59:55 2011 From: thomas.witkowski at tu-dresden.de (Thomas Witkowski) Date: Fri, 20 May 2011 15:59:55 +0200 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: References: <4DD664B7.6030308@tu-dresden.de> <4DD669A5.3080606@tu-dresden.de> Message-ID: <4DD673DB.1020900@tu-dresden.de> Jed Brown wrote: > On Fri, May 20, 2011 at 15:16, Thomas Witkowski > > wrote: > > Thanks for the information. One related question: The system that > is solved here is a 3x3 block matrix: Two of these blocks are > empty, two are the discretized (FEM) identity, three are the > discretized Laplace operator > > > What equation? Have you assumed special boundary conditions? Problems > like elasticity and Stokes normally need a symmetric stress tensor. > Only for constant coefficients and certain boundary conditions, is it > okay to use the "block Laplacian" form. It is the so called phase field crystal equation (PFC, a 6th-order nonlinear parabolic PDE). The block structure results because my code can solve only system of 2nd-order PDEs. Because there is a Laplace^3 in the original equation I get all the Laplacian due to operator spliting. The whole block looks as follows: [-L, 0, I; I, -L, 0; C, I, -L], where I is the identity operator, L the Laplace operator and C = 2 L - eps I + v I, with eps a number and v the solution of the old timestep). There are many different ways to write the system. This here is not symmetric, but is more diagonal dominant. Boundary conditions are zero-flux on all boundaries. > > > and one block is the discretization of a complex sum of operators. > At all, there are only three different blocks. Can I make somehow > advantages of this structure? I could create the three inverses of > the three different blocks, but exists than a cheap/simple way to > produce the action of the inverse of the whole 3x3 block matrix? > > > For the special case of a Kronecker product, you can solve using > solves with the blocks, but not in the general case. Consider the > structured grid 2D Laplacian [A, -I; -I, A] where A is the tridiagonal > problem for a 1D Laplacian plus 2*I on the diagonal. If you could > build a direct solver for the 2D using direct solvers for the 1D > problem, then you would have an O(n) algorithm for a direct solve of a > 2D Laplacian. > > You can, however, build an iterative method from the blocks. If each > block is built separately, you can (with petsc-dev) put them into > MatNest and use PCFieldSplit with direct solves in splits. I will check this. > > An alternative is to interlace the degrees of freedom and use the > (S)BAIJ matrix format. This will reduce the amount of memory needed > for your matrix (by using a more efficient storage format) and will > improve the memory performance. Unfortunately, there are no > third-party direct solvers for BAIJ. Note that your assembly code can > always call MatSetValuesBlocked() (or ..BlockedLocal()) regardless of > the matrix format, so you can switch between AIJ and BAIJ at run-time. > > Why bother with a direct solver if your problem has constant > coefficients? From bsmith at mcs.anl.gov Fri May 20 09:06:56 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 20 May 2011 09:06:56 -0500 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> <6260C6F6-13F6-48BA-B1A6-F1FFE2C9FB46@mcs.anl.gov> Message-ID: <5C79D8AC-8015-4D89-96AE-C1B34EEDFE8A@mcs.anl.gov> Be sure you are using the same definition of interlaced as we are. Generally when interlacing inodes will exist, when not interlacing they will not exist. From you print outs you have the opposite effect which is unlikely. Say we have a PDE with three components at each grid point, u, v, and p. Then interlaced is storing the values [u0 v0 p0 u1 v1 p1 ....] noninterlaced is storing it [u0 u1 ... v0 v1 .... p0 p1 ....] Barry On May 20, 2011, at 1:31 AM, Henning Sauerland wrote: > > Am 18.05.2011 um 19:46 schrieb Barry Smith: >> >> So interlacing the variables makes ILU() much worse in both iteration count and time? > > > Yes. I guess the reason is that the inode routines are not used with the interlaced ordering: > > interlaced: > KSP Object: > type: lgmres > GMRES: restart=200, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > LGMRES: aug. dimension=2 > LGMRES: number of matvecs=592 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-08, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: > type: asm > Additive Schwarz: total subdomain blocks = 16, amount of overlap = 1 > Additive Schwarz: restriction/interpolation type - RESTRICT > Local solve is same for all blocks, in the following KSP and PC objects: > KSP Object:(sub_) > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object:(sub_) > type: ilu > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 1e-12 > using diagonal shift to prevent zero pivot > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Matrix Object: > type=seqaij, rows=144120, cols=144120 > package used to perform factorization: petsc > total: nonzeros=14238265, allocated nonzeros=14238265 > not using I-node routines > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=144120, cols=144120 > total: nonzeros=14238265, allocated nonzeros=14238265 > not using I-node routines > linear system matrix = precond matrix: > Matrix Object: > type=mpiaij, rows=548908, cols=548908 > total: nonzeros=55971327, allocated nonzeros=93314360 > not using I-node (on process 0) routines > > > non-interlaced: > KSP Object: > type: lgmres > GMRES: restart=200, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > LGMRES: aug. dimension=2 > LGMRES: number of matvecs=334 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-08, absolute=1e-50, divergence=10000 > left preconditioning > diagonally scaled system > using PRECONDITIONED norm type for convergence test > PC Object: > type: asm > Additive Schwarz: total subdomain blocks = 16, amount of overlap = 1 > Additive Schwarz: restriction/interpolation type - RESTRICT > Local solve is same for all blocks, in the following KSP and PC objects: > KSP Object:(sub_) > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object:(sub_) > type: ilu > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 1e-12 > using diagonal shift to prevent zero pivot > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Matrix Object: > type=seqaij, rows=41200, cols=41200 > package used to perform factorization: petsc > total: nonzeros=3651205, allocated nonzeros=3651205 > using I-node routines: found 15123 nodes, limit used is 5 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=41200, cols=41200 > total: nonzeros=3651205, allocated nonzeros=3651205 > using I-node routines: found 15123 nodes, limit used is 5 > linear system matrix = precond matrix: > Matrix Object: > type=mpiaij, rows=548908, cols=548908 > total: nonzeros=55971327, allocated nonzeros=112526140 > using I-node (on process 0) routines: found 13156 nodes, limit used is 5 > > From bsmith at mcs.anl.gov Fri May 20 09:14:26 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 20 May 2011 09:14:26 -0500 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: <4DD664B7.6030308@tu-dresden.de> References: <4DD664B7.6030308@tu-dresden.de> Message-ID: <6B919430-015D-438E-AC34-6ABD3AA2B928@mcs.anl.gov> On May 20, 2011, at 7:55 AM, Thomas Witkowski wrote: > Could one of you explain me why direct solvers (I make use of UMFPACK) seems to work quite bad for 3D-FEM? For a small test case, I take a unite square/box and refine it globally (bisectioning of triangle/tetrahedrons). I solve a six order PDE (that leads to symmetric but indefinite matrices) on it. In 2D the resulting system has 49.923 rows with 808.199 non zeros, UMFPACK solves the system within 2.8 seconds. For 3D I've refine the box such that the resulting matrix has more or less the same number of non zeros, in this case 898.807 on 27.027 rows. UMFPACK needs 32 seconds to solve it, so more than 10 time as for the 2D case. Even if I make use of fourth order Lagrange functions for the 2D case, which leads to much denser matrices (49.923 rows and 2.705.927 non zeros), UMFPACK solves it within 3.2 seconds. Is there any good reason for it? > > Thomas Thomas, This difference was originally explained by Alan George in seminal work on sparse direct solvers in the late 60s and early 70s. He has several publications that explain the reason for the difference and a book (which just happens to be online at http://www.cse.illinois.edu/courses/cs598mh/george_liu.pdf). This difference between 2 and 3 d direct solvers is just a fact of life. Barry From jed at 59A2.org Fri May 20 09:23:29 2011 From: jed at 59A2.org (Jed Brown) Date: Fri, 20 May 2011 16:23:29 +0200 Subject: [petsc-users] Direct solver for 3D-FEM In-Reply-To: <6B919430-015D-438E-AC34-6ABD3AA2B928@mcs.anl.gov> References: <4DD664B7.6030308@tu-dresden.de> <6B919430-015D-438E-AC34-6ABD3AA2B928@mcs.anl.gov> Message-ID: On Fri, May 20, 2011 at 16:14, Barry Smith wrote: > and a book (which just happens to be online at > http://www.cse.illinois.edu/courses/cs598mh/george_liu.pdf) > Cool, I've looked for this before and couldn't find an electronic copy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.witkowski at tu-dresden.de Mon May 23 06:25:46 2011 From: thomas.witkowski at tu-dresden.de (Thomas Witkowski) Date: Mon, 23 May 2011 13:25:46 +0200 Subject: [petsc-users] configure of PETSc with multiple cores Message-ID: <4DDA443A.2020305@tu-dresden.de> Is there a way to run ./configure with multiple cores? On my systems it takes up to three hours to run this script. So around 80% of the time is spend in compiling third party packages. Is it possible to run at least this compilation with multiple cores, so to provide something like "-j8" to them? Thomas From jed at 59A2.org Mon May 23 06:39:23 2011 From: jed at 59A2.org (Jed Brown) Date: Mon, 23 May 2011 13:39:23 +0200 Subject: [petsc-users] configure of PETSc with multiple cores In-Reply-To: <4DDA443A.2020305@tu-dresden.de> References: <4DDA443A.2020305@tu-dresden.de> Message-ID: On Mon, May 23, 2011 at 13:25, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > Is there a way to run ./configure with multiple cores? On my systems it > takes up to three hours to run this script. So around 80% of the time is > spend in compiling third party packages. Is it possible to run at least this > compilation with multiple cores, so to provide something like "-j8" to them? Not currently. The most common reason for slow configure is a slow network-mounted disk. You can configure on a scratch disk (set symlinks from where you want to put it). With petsc-dev, you can run multiple configures concurrently (for different PETSC_ARCH). -------------- next part -------------- An HTML attachment was scrubbed... URL: From uerland at gmail.com Mon May 23 06:58:27 2011 From: uerland at gmail.com (Henning Sauerland) Date: Mon, 23 May 2011 13:58:27 +0200 Subject: [petsc-users] Tuning the parallel performance of a 3D FEM CFD code In-Reply-To: <5C79D8AC-8015-4D89-96AE-C1B34EEDFE8A@mcs.anl.gov> References: <88D8A52D-018F-4BF2-9DB5-F9B4410F98F7@gmail.com> <36779024-BEB1-4C10-B1F8-612360AC51FF@gmail.com> <713506C4-225D-4851-8C53-C3E2B3627878@gmail.com> <4EEF8D1C-8F8A-4ECF-887B-2F00A9599A61@gmail.com> <6260C6F6-13F6-48BA-B1A6-F1FFE2C9FB46@mcs.anl.gov> <5C79D8AC-8015-4D89-96AE-C1B34EEDFE8A@mcs.anl.gov> Message-ID: Thanks for the clarification. I mixed it up. On 20.05.2011, at 16:06, Barry Smith wrote: > > Be sure you are using the same definition of interlaced as we are. Generally when interlacing inodes will exist, when not interlacing they will not exist. From you print outs you have the opposite effect which is unlikely. > > Say we have a PDE with three components at each grid point, u, v, and p. Then interlaced is storing the values [u0 v0 p0 u1 v1 p1 ....] noninterlaced is storing it [u0 u1 ... v0 v1 .... p0 p1 ....] From wgropp at illinois.edu Mon May 23 09:44:27 2011 From: wgropp at illinois.edu (William Gropp) Date: Mon, 23 May 2011 09:44:27 -0500 Subject: [petsc-users] Examples needed for testing a new sparse matrix format Message-ID: <47356F3D-58F5-4EDF-A2C4-36B0E87D5280@illinois.edu> We are looking for some example applications that we could use to test a new sparse matrix format. This format gives significantly better performance on some test cases, but we'd like to test it in some applications. The new format is described in http://hpc.sagepub.com/content/25/1/115.short . Let us know if you have any suggestions for us. Thanks! Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign From charlesreid1 at gmail.com Mon May 23 14:16:39 2011 From: charlesreid1 at gmail.com (charles reid) Date: Mon, 23 May 2011 13:16:39 -0600 Subject: [petsc-users] configure mpirun problem Message-ID: Hi - I'm trying to install Petsc and linking to OpenMPI, but running into a conflict with the compiler test. If I specify my MPI installation directory and my mpiexec command, like so: ./configure \ --prefix=/uufs/chpc.utah.edu/common/home/u0552682/pkg/petsc/3.0.0_incendio\ --with-x=false \ --with-matlab=false \ --download-f-blas-lapack=ifneeded \ \ --with-mpi-dir=/uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3 \ --with-mpiexec=/uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpirun \ \ --with-blas-lapack-dir=/usr/lib \Checking for program /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc...found Defined make macro "CC" to "/uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc" Pushing language C sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c Executing: /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c sh: sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c Executing: /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c sh: Pushing language C Popping language C Pushing language Cxx Popping language Cxx Pushing language FC Popping language FC Pushing language C Popping language C sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o conftest conftest.o Executing: /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o conftest conftest.o sh: sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c Executing: /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o conftest.o conftest.c sh: Pushing language C Popping language C sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o conftest conftest.o Executing: /uufs/ chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o conftest conftest.o sh:Executing: ./conftest sh: ./conftest Executing: ./conftestsh: ERROR while running executable: Could not execute './conftest':./conftest: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory --with-shared=0 \ The underlying problem can be demonstrated with a simple C hello world program: $ cat test.c #include main() { printf ("Hello World!\n"); } $ /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o test.o test.c $ ./test.o ./test.o: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory Running the executable with mpirun alone shows the same error (similar to this thread at OpenMPI list: http://www.open-mpi.org/community/lists/users/2006/07/1691.php) $ mpirun -np 1 ./test.o ./test.o: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory and ultimately the resolution is to specify the full path of mpirun when running the executable (as per the response to the above thread, http://www.open-mpi.org/community/lists/users/2006/07/1692.php): $ `which mpirun` -np 1 ./test.o Hello World! So my question is, why does Petsc's compiler test insist on running the mpicc-built test executable in an incorrect way (i.e. ./conftestsh instead of /path/to/mpiexec ./conftest)? And how can I compile with OpenMPI support? I'm unable to bypass this by specifying CC and CXX, as I just run into problems later on with mpi_init not being found. Charles -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 23 14:21:09 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 23 May 2011 14:21:09 -0500 Subject: [petsc-users] configure mpirun problem In-Reply-To: References: Message-ID: On Mon, May 23, 2011 at 2:16 PM, charles reid wrote: > Hi - > > I'm trying to install Petsc and linking to OpenMPI, but running into a > conflict with the compiler test. If I specify my MPI installation directory > and my mpiexec command, like so: > > ./configure \ > --prefix=/uufs/ > chpc.utah.edu/common/home/u0552682/pkg/petsc/3.0.0_incendio \ > --with-x=false \ > --with-matlab=false \ > --download-f-blas-lapack=ifneeded \ > \ > --with-mpi-dir=/uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3\ > --with-mpiexec=/uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpirun \ > \ > --with-blas-lapack-dir=/usr/lib \Checking for program /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc...found > Defined make macro "CC" to "/uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc" > Pushing language C > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > Executing: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > conftest.o conftest.c > sh: > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > Executing: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > conftest.o conftest.c > sh: > Pushing language C > Popping language C > Pushing language Cxx > Popping language Cxx > Pushing language FC > Popping language FC > Pushing language C Popping language C > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > -o conftest conftest.o > Executing: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > conftest conftest.o > sh: > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > Executing: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > conftest.o conftest.c > sh: Pushing language C > Popping language C > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > -o conftest conftest.o > Executing: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > conftest conftest.o > sh:Executing: ./conftest > sh: ./conftest > Executing: ./conftestsh: > ERROR while running executable: Could not execute './conftest':./conftest: > error while loading shared libraries: libmpi.so.0: cannot open shared object > file: No such file or directory > --with-shared=0 \ > > > > The underlying problem can be demonstrated with a simple C hello world > program: > > $ cat test.c > > #include > > main() > { > printf ("Hello World!\n"); > } > > $ /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > test.o test.c > > $ ./test.o > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open > shared object file: No such file or directory > > > > Running the executable with mpirun alone shows the same error (similar to > this thread at OpenMPI list: > http://www.open-mpi.org/community/lists/users/2006/07/1691.php) > > $ mpirun -np 1 ./test.o > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open > shared object file: No such file or directory > > > > and ultimately the resolution is to specify the full path of mpirun when > running the executable (as per the response to the above thread, > http://www.open-mpi.org/community/lists/users/2006/07/1692.php): > > $ `which mpirun` -np 1 ./test.o > Hello World! > > > > So my question is, why does Petsc's compiler test insist on running the > mpicc-built test executable in an incorrect way (i.e. ./conftestsh instead > of /path/to/mpiexec ./conftest)? And how can I compile with OpenMPI > support? I'm unable to bypass this by specifying CC and CXX, as I just run > into problems later on with mpi_init not being found. There are many, many ways to run an executable. We count on 'sh ' as a default, and if that does not work then you can run configure using --with-batch. Matt > > Charles > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Mon May 23 14:23:37 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 23 May 2011 14:23:37 -0500 (CDT) Subject: [petsc-users] configure mpirun problem In-Reply-To: References: Message-ID: On Mon, 23 May 2011, Matthew Knepley wrote: > On Mon, May 23, 2011 at 2:16 PM, charles reid wrote: > > > Hi - > > > > I'm trying to install Petsc and linking to OpenMPI, but running into a > > conflict with the compiler test. If I specify my MPI installation directory > > and my mpiexec command, like so: > > > > ./configure \ > > --prefix=/uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/petsc/3.0.0_incendio \ > > --with-x=false \ > > --with-matlab=false \ > > --download-f-blas-lapack=ifneeded \ > > \ > > --with-mpi-dir=/uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3\ > > --with-mpiexec=/uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpirun \ > > \ > > --with-blas-lapack-dir=/usr/lib \Checking for program /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc...found > > Defined make macro "CC" to "/uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc" > > Pushing language C > > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > > Executing: /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > conftest.o conftest.c > > sh: > > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > > Executing: /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > conftest.o conftest.c > > sh: > > Pushing language C > > Popping language C > > Pushing language Cxx > > Popping language Cxx > > Pushing language FC > > Popping language FC > > Pushing language C Popping language C > > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > > -o conftest conftest.o > > Executing: /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > conftest conftest.o > > sh: > > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o conftest.c > > Executing: /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > conftest.o conftest.c > > sh: Pushing language C > > Popping language C > > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > > -o conftest conftest.o > > Executing: /uufs/ > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > conftest conftest.o > > sh:Executing: ./conftest > > sh: ./conftest > > Executing: ./conftestsh: > > ERROR while running executable: Could not execute './conftest':./conftest: > > error while loading shared libraries: libmpi.so.0: cannot open shared object > > file: No such file or directory > > --with-shared=0 \ > > > > > > > > The underlying problem can be demonstrated with a simple C hello world > > program: > > > > $ cat test.c > > > > #include > > > > main() > > { > > printf ("Hello World!\n"); > > } > > > > $ /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > test.o test.c > > > > $ ./test.o > > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open > > shared object file: No such file or directory > > > > > > > > Running the executable with mpirun alone shows the same error (similar to > > this thread at OpenMPI list: > > http://www.open-mpi.org/community/lists/users/2006/07/1691.php) > > > > $ mpirun -np 1 ./test.o > > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open > > shared object file: No such file or directory > > > > > > > > and ultimately the resolution is to specify the full path of mpirun when > > running the executable (as per the response to the above thread, > > http://www.open-mpi.org/community/lists/users/2006/07/1692.php): > > > > $ `which mpirun` -np 1 ./test.o > > Hello World! > > > > > > > > So my question is, why does Petsc's compiler test insist on running the > > mpicc-built test executable in an incorrect way (i.e. ./conftestsh instead > > of /path/to/mpiexec ./conftest)? And how can I compile with OpenMPI > > support? I'm unable to bypass this by specifying CC and CXX, as I just run > > into problems later on with mpi_init not being found. > > > There are many, many ways to run an executable. We count on 'sh > ' as a default, and if that does not work > then you can run configure using --with-batch. With openMPI - the primary issue is: They [openmpi folks] expect openmpi users to set LD_LIBRARY_PATH before using their compilers. So you should set that before running PETSc configure. Also - you are using older 3.0.0 version. 3.1 might have some workarrounds for openmpi [for sure petsc-dev does..] Satish > > Matt > > > > > > Charles > > > > > > From charlesreid1 at gmail.com Mon May 23 15:53:38 2011 From: charlesreid1 at gmail.com (charles reid) Date: Mon, 23 May 2011 14:53:38 -0600 Subject: [petsc-users] configure mpirun problem In-Reply-To: References: Message-ID: Hi - Thanks for the responses. Setting LD_LIBRARY_PATH has no effect on the process, since the problem isn't with LD_LIBRARY_PATH or the compilation process, its with how the executable is being run. I also tried Petsc 3.1 but got the same problem (with and without LD-LIBRARY-PATH set). The --with-batch flag ended up being the solution that worked, once configure stopped I was able to run conftest through mpirun and then run reconfigure.py, and everything worked as expected. Thanks again for the help! Charles On Mon, May 23, 2011 at 13:23, Satish Balay wrote: > On Mon, 23 May 2011, Matthew Knepley wrote: > > > On Mon, May 23, 2011 at 2:16 PM, charles reid >wrote: > > > > > Hi - > > > > > > I'm trying to install Petsc and linking to OpenMPI, but running into a > > > conflict with the compiler test. If I specify my MPI installation > directory > > > and my mpiexec command, like so: > > > > > > ./configure \ > > > --prefix=/uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/petsc/3.0.0_incendio \ > > > --with-x=false \ > > > --with-matlab=false \ > > > --download-f-blas-lapack=ifneeded \ > > > \ > > > --with-mpi-dir=/uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3\ > > > --with-mpiexec=/uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpirun \ > > > \ > > > --with-blas-lapack-dir=/usr/lib \Checking for program /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc...found > > > Defined make macro "CC" to "/uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc" > > > Pushing language C > > > sh: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o > conftest.o conftest.c > > > Executing: /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > > conftest.o conftest.c > > > sh: > > > sh: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o > conftest.o conftest.c > > > Executing: /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > > conftest.o conftest.c > > > sh: > > > Pushing language C > > > Popping language C > > > Pushing language Cxx > > > Popping language Cxx > > > Pushing language FC > > > Popping language FC > > > Pushing language C Popping language > C > > > sh: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > > > -o conftest conftest.o > > > Executing: /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > > conftest conftest.o > > > sh: > > > sh: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o > conftest.o conftest.c > > > Executing: /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o > > > conftest.o conftest.c > > > sh: Pushing language C > > > Popping language C > > > sh: /uufs/ > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc > > > -o conftest conftest.o > > > Executing: /uufs/ > > > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > > conftest conftest.o > > > sh:Executing: ./conftest > > > sh: ./conftest > > > Executing: ./conftestsh: > > > ERROR while running executable: Could not execute > './conftest':./conftest: > > > error while loading shared libraries: libmpi.so.0: cannot open shared > object > > > file: No such file or directory > > > --with-shared=0 \ > > > > > > > > > > > > The underlying problem can be demonstrated with a simple C hello world > > > program: > > > > > > $ cat test.c > > > > > > #include > > > > > > main() > > > { > > > printf ("Hello World!\n"); > > > } > > > > > > $ /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -o > > > test.o test.c > > > > > > $ ./test.o > > > ./test.o: error while loading shared libraries: libmpi.so.0: cannot > open > > > shared object file: No such file or directory > > > > > > > > > > > > Running the executable with mpirun alone shows the same error (similar > to > > > this thread at OpenMPI list: > > > http://www.open-mpi.org/community/lists/users/2006/07/1691.php) > > > > > > $ mpirun -np 1 ./test.o > > > ./test.o: error while loading shared libraries: libmpi.so.0: cannot > open > > > shared object file: No such file or directory > > > > > > > > > > > > and ultimately the resolution is to specify the full path of mpirun > when > > > running the executable (as per the response to the above thread, > > > http://www.open-mpi.org/community/lists/users/2006/07/1692.php): > > > > > > $ `which mpirun` -np 1 ./test.o > > > Hello World! > > > > > > > > > > > > So my question is, why does Petsc's compiler test insist on running the > > > mpicc-built test executable in an incorrect way (i.e. ./conftestsh > instead > > > of /path/to/mpiexec ./conftest)? And how can I compile with OpenMPI > > > support? I'm unable to bypass this by specifying CC and CXX, as I just > run > > > into problems later on with mpi_init not being found. > > > > > > There are many, many ways to run an executable. We count on 'sh > > ' as a default, and if that does not work > > then you can run configure using --with-batch. > > With openMPI - the primary issue is: > > They [openmpi folks] expect openmpi users to set LD_LIBRARY_PATH > before using their compilers. So you should set that before running > PETSc configure. > > Also - you are using older 3.0.0 version. 3.1 might have some > workarrounds for openmpi [for sure petsc-dev does..] > > Satish > > > > > Matt > > > > > > > > > > Charles > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Tue May 24 07:06:14 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 24 May 2011 20:06:14 +0800 (CST) Subject: [petsc-users] Flexiable AIJ matrix for nonzeros -- hash table version In-Reply-To: References: <22999979.13081304931864812.JavaMail.coremail@mail.ustc.edu> <16620186.26441305549966454.JavaMail.coremail@mail.ustc.edu> <14631630.27291305598579219.JavaMail.coremail@mail.ustc.edu> Message-ID: <25649841.41341306238774586.JavaMail.coremail@mail.ustc.edu> Flexiable AIJ matrix for nonzeros -- hash table version. Both patch file and hg bundle file to latest petsc-dev are attached. I had only tested c version. Fortran interface is NOT tested! -------------- next part -------------- A non-text attachment was scrubbed... Name: mat_aij_hashtable.hg Type: application/octet-stream Size: 14836 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mat_aij_hashtable.patch Type: application/octet-stream Size: 100891 bytes Desc: not available URL: From bartlomiej.wach at yahoo.pl Tue May 24 14:16:07 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Tue, 24 May 2011 20:16:07 +0100 (BST) Subject: [petsc-users] PETSc on 64 bit machine Message-ID: <256828.60038.qm@web28316.mail.ukl.yahoo.com> Hello, Trying to configure PETSc on 64 bit Ubuntu with: ./config/configure.py --with-cc=gcc --with-cxx=g++ --with-fc=0 --with-mpi=0 --with-64-bit-pointers=1 --with-64-bit-indices=1 getting Failed to find 8-bit clean memcmp(). Cannot proceed. Did anyone meet with this issue? Regards Bartek Wach -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: text/x-log Size: 2034 bytes Desc: not available URL: From balay at mcs.anl.gov Tue May 24 14:56:40 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 24 May 2011 14:56:40 -0500 (CDT) Subject: [petsc-users] PETSc on 64 bit machine In-Reply-To: <256828.60038.qm@web28316.mail.ukl.yahoo.com> References: <256828.60038.qm@web28316.mail.ukl.yahoo.com> Message-ID: Is gcc functional on this machine? A simple compile by configure gives a wierd error. Satish >>>>>> sh: gcc -o conftest -m64 -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -ldl Executing: gcc -o conftest -m64 -Wall -Wwrite-strings -Wno-strict-aliasing -g3 conftest.o -ldl sh: Executing: ./conftest sh: ./conftest Executing: ./conftest sh: ERROR while running executable: Could not execute './conftest': ./conftest: 8: Syntax error: "(" unexpected <<<<<< On Tue, 24 May 2011, Bart?omiej W wrote: > Hello, > > Trying to configure PETSc on 64 bit Ubuntu with: > > ./config/configure.py --with-cc=gcc --with-cxx=g++ --with-fc=0 --with-mpi=0 --with-64-bit-pointers=1 --with-64-bit-indices=1 > > getting > > Failed to find 8-bit clean memcmp(). Cannot proceed. > > Did anyone meet with this issue? > > Regards > Bartek Wach > From bartlomiej.wach at yahoo.pl Tue May 24 17:14:44 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Tue, 24 May 2011 23:14:44 +0100 (BST) Subject: [petsc-users] PETSc on 64 bit machine In-Reply-To: Message-ID: <943211.43187.qm@web28310.mail.ukl.yahoo.com> Clean reinstall and an update of ubuntu was a solution, sorry for the spam. --- 24.5.11 (Wt), Satish Balay napisa?(a): Od: Satish Balay Temat: Re: [petsc-users] PETSc on 64 bit machine Do: "PETSc users list" Data: 24 Maj 2011 (Wtorek), 19:56 Is gcc functional on this machine? A simple compile by configure gives a wierd error. Satish >>>>>> sh: gcc? -o conftest? ? -m64 -Wall -Wwrite-strings -Wno-strict-aliasing -g3? conftest.o -ldl Executing: gcc? -o conftest? ? -m64 -Wall -Wwrite-strings -Wno-strict-aliasing -g3? conftest.o -ldl sh: Executing: ./conftest sh: ./conftest Executing: ./conftest sh: ERROR while running executable: Could not execute './conftest': ./conftest: 8: Syntax error: "(" unexpected <<<<<< On Tue, 24 May 2011, Bart?omiej W wrote: > Hello, > > Trying to configure PETSc on 64 bit Ubuntu with: > > ./config/configure.py --with-cc=gcc --with-cxx=g++ --with-fc=0 --with-mpi=0 --with-64-bit-pointers=1 --with-64-bit-indices=1 > > getting > > Failed to find 8-bit clean memcmp(). Cannot proceed. > > Did anyone meet with this issue? > > Regards > Bartek Wach > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Wed May 25 02:45:43 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Wed, 25 May 2011 09:45:43 +0200 Subject: [petsc-users] ILU and Block Gauss Seidel smoothing Message-ID: Dear all, I am writing a multigrid solver for the Navier Stokes Equations, using PETSc. I am not using the dmmg structure because I need grid refinement and, as far as I can see, that is not supported. I have two questions concerning two different possibilities of smoothing: 1) ILU relaxation: in Multigrid (U. Trottenberg,Cornelis W. Oosterlee,Anton Sch?ller), it is suggested that alternating ILU represents a robust smoother. Given a routine for ILU decomposition, my understanding is that alternation depends on the ordering of the matrix. My code builds a linearized Navier Stokes operator where first the u momentum equation, then the v momentum and then the p equation are stored. Each equation is built in EN order ? first comes the bottom horizontal line, from West to East, then I move line by line from South to North. Otherwise stated, the position of each unknown in the matrix is given by P = cf*nx*ny + j*nx + i, where cf is the variable number (0 = u, 1 = v, 2 = p) and i,j,nx,ny are as usual. For "historical" reasons I'm not using the DM structure even if the grids are logically rectangular, but I can change that. My question: is there in PETSc a way of doing alternating ILU which does not require rebuilding the matrix with different ordering each time? Somehow related questions: what is the best ordering for the matrix and which one is the one used by the DM structure? 2) Decoupled block-line Gauss Seidel relaxation: this is another smoother I'm considering, which explains the matrix ordering above (I use block horizontal lines smoothing, so that each line is stored consecutively in the matrix). At the moment the full linearized operator is first built and then parts of the matrix are extracted. In reality though, I never need the full linearized operator to be built except on the coarsest grid. Rather, I need it to be constructed each block-line at a time. I've seen that for parallel applications each part of the matrix is built on its corresponding processor. Is there a way to do it sequentially and to have control on which part is built? Any other suggestion on both subjects is of course welcome. Thanks in advance Gianluca -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From miguel.fosas at gmail.com Wed May 25 05:38:45 2011 From: miguel.fosas at gmail.com (Miguel Fosas) Date: Wed, 25 May 2011 12:38:45 +0200 Subject: [petsc-users] Undefined symbols 'petscstack' and 'PetscMemzero' Message-ID: Dear all, I'm trying to compile PETSc (3.1-p8) on an SGI Altix using Intel C/C++ compilers (the exact configure line is shown below). When trying to perform the tests, the compilation of the examples fails because the linker is not able to find the symbols 'petscstack' and "PetscMemzero'. On top of that, nm returns: $ nm $PETSC_DIR/lib/*.so | grep PetscMemzero | c++filt U PetscMemzero(void*, unsigned long) $ nm $PETSC_DIR/lib/*.so | grep petscstack U petscstack The output of nm is the same for *.a The compilation of the examples returns: ~/local/petsc/lib/libpetsc.so: undefined reference to `petscstack' ~/local/petsc/lib/libpetsc.so: undefined reference to `PetscMemzero(void*, unsigned long)' and the configuration line is: ./configure --prefix=$HOME/local/petsc --with-clanguage=c++ --with-mpi=yes --with-mpi-shared=1 --with-scalar-type=complex --with-fortran --with-fortran-kernels=1 --with-cc=icc --with-fc=ifort --with-cxx=icpc --with-debugging=0 --with-blas-lapack-dir=/usr/local/intel/mkl/ 10.2.2.025/lib/64 --with-shared=1 -with-x=1 --with-batch=1 --known-mpi-shared=0 --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-hdf5-dir=$HOME/local Is there any option that might be causing this problem? Thanks, Miguel. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aron.ahmadia at kaust.edu.sa Wed May 25 05:54:07 2011 From: aron.ahmadia at kaust.edu.sa (Aron Ahmadia) Date: Wed, 25 May 2011 13:54:07 +0300 Subject: [petsc-users] Undefined symbols 'petscstack' and 'PetscMemzero' In-Reply-To: References: Message-ID: Hi Miguel, Please send the complete configure.log and make.log to petsc-maint at mcs.anl.gov Are you able to build with --with-shared=0? A On Wed, May 25, 2011 at 1:38 PM, Miguel Fosas wrote: > Dear all, > > I'm trying to compile PETSc (3.1-p8) on an SGI Altix using Intel C/C++ > compilers (the exact configure line is shown below). When trying to perform > the tests, the compilation of the examples fails because the linker is not > able to find the symbols 'petscstack' and "PetscMemzero'. On top of that, nm > returns: > > $ nm $PETSC_DIR/lib/*.so | grep PetscMemzero | c++filt > U PetscMemzero(void*, unsigned long) > > $ nm $PETSC_DIR/lib/*.so | grep petscstack > U petscstack > > The output of nm is the same for *.a > > The compilation of the examples returns: > ~/local/petsc/lib/libpetsc.so: undefined reference to `petscstack' > ~/local/petsc/lib/libpetsc.so: undefined reference to `PetscMemzero(void*, > unsigned long)' > > and the configuration line is: > > ./configure --prefix=$HOME/local/petsc --with-clanguage=c++ --with-mpi=yes > --with-mpi-shared=1 --with-scalar-type=complex --with-fortran > --with-fortran-kernels=1 --with-cc=icc --with-fc=ifort --with-cxx=icpc > --with-debugging=0 --with-blas-lapack-dir=/usr/local/intel/mkl/ > 10.2.2.025/lib/64 --with-shared=1 -with-x=1 --with-batch=1 > --known-mpi-shared=0 --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 > --with-hdf5-dir=$HOME/local > > Is there any option that might be causing this problem? > > Thanks, > > Miguel. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourdin at lsu.edu Wed May 25 10:05:56 2011 From: bourdin at lsu.edu (Blaise Bourdin) Date: Wed, 25 May 2011 10:05:56 -0500 Subject: [petsc-users] tecplot viewer Message-ID: Hi, Has anybody in this list ever written a tecplot viewer for DA based data? I have been fighting with tecplot hdf5 reader and it seems like it is not capable of reading vector valued fields. Blaise -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin From lizs at mail.uc.edu Wed May 25 18:42:33 2011 From: lizs at mail.uc.edu (Li, Zhisong (lizs)) Date: Wed, 25 May 2011 23:42:33 +0000 Subject: [petsc-users] Petsc option and scalablity problems on ML Message-ID: Hi, Petsc Team, I have been working for a while on applying ML precoditioner to improve performance of a KSP solving a Poisson-style problem. So far the only speedup gain attributed to resetting the number of multigrid levels. I wonder if all the available Petsc options to control ML are limited to those listed on the Petsc manual (about 10 totally). Do we have any other controls more than that? In ML user's guide, they state another tip to improve ML's parallel speed: "Instead of doing a direct solve on the coarsest level, try a few smoothing sweeps instead". I don't think the available options "-pc_mg_smoothup" and "-pc_mg_smoothdown" correspond to this control. Can any PETSC interface option do that? The domain has about 1.2 M grid points. As limited by the machine's RAM, I cannot further increase the problem size at this time. The parallel performance reaches its peak at only 8 nodes. It takes ML and GMRES about 20 s to converge to a relative tol of 10e-6 for each time step, compared with 74s without any PC. But 20s is still unacceptably large for an unsteady simulation which may need thousands of steps. Any suggestions to improve this scalability? Thank you very much. Sincerely, Zhisong Li -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Wed May 25 18:59:06 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 26 May 2011 01:59:06 +0200 Subject: [petsc-users] Petsc option and scalablity problems on ML In-Reply-To: References: Message-ID: On Thu, May 26, 2011 at 01:42, Li, Zhisong (lizs) wrote: > I have been working for a while on applying ML precoditioner to improve > performance of a KSP solving a Poisson-style problem. So far the only > speedup gain attributed to resetting the number of multigrid levels. I > wonder if all the available Petsc options to control ML are limited to those > listed on the Petsc manual (about 10 totally). Do we have any other controls > more than that? > Run with "-help |grep pc_ml_" to see the full list of options that can be passed to ML. ML constructs the hierarchy and then you can control it through PCMG. Relevant options are under the prefixes -pc_mg_*, -mg_coarse_*, and -mg_levels_*. > In ML user's guide, they state another tip to improve ML's parallel speed: > "Instead of doing a direct solve on the coarsest level, try a few smoothing > sweeps instead". I don't think the available options "-pc_mg_smoothup" and > "-pc_mg_smoothdown" correspond to this control. Can any PETSC interface > option do that? > You can control the coarse level with -mg_coarse_ksp_*, for example, -mg_coarse_ksp_type cg -mg_coarse_ksp_max_it 3 -mg_coarse_pc_type bjacobi -mg_coarse_sub_pc_type lu to do 3 cycles of CG preconditioned by block Jacobi with direct subdomain solves (trivially cheap on a coarse level) as the coarse level solver. Note that Krylov iterations here will make the preconditioner nonlinear so you should use -ksp_type fgmres on the outside. If you know something about the spectrum, you could use something like -mg_coarse_ksp_type chebychev which is linear for a fixed number of iterations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Wed May 25 19:48:07 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 26 May 2011 01:48:07 +0100 (BST) Subject: [petsc-users] Insering rows vs inserting columns In-Reply-To: Message-ID: <987901.42592.qm@web28305.mail.ukl.yahoo.com> Hello, I have a slight problem with performance of matsetvalues when inserting values row-by-row on single processor instead of column-by-column. ???? ierr = MatCreate(PETSC_COMM_WORLD,&L); ???? ierr = MatSetSizes(L,PETSC_DECIDE,PETSC_DECIDE,n,n); ???? ierr = MatSetFromOptions(L); ???? MatSeqAIJSetPreallocation(L,PETSC_NULL,nnz); ? ? (...) ??? ???? MatSetValues(L,rows,indices_y,1,&col,values,INSERT_VALUES); vs ??? ???? MatSetValues(L,1,&row,cols,indices_x,values,INSERT_VALUES); First way goes quick and gives me mallocs in matsetvalues = 0 in -info While the second one takes much longer and does mallocs. Why is it so that inserting rows is slower than inserting columns when proper nnz was provided? Or maybe there is my mistake here somewhere? Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed May 25 21:25:37 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 25 May 2011 21:25:37 -0500 (CDT) Subject: [petsc-users] Insering rows vs inserting columns In-Reply-To: <987901.42592.qm@web28305.mail.ukl.yahoo.com> References: <987901.42592.qm@web28305.mail.ukl.yahoo.com> Message-ID: On Thu, 26 May 2011, Bart?omiej W wrote: > Hello, > > I have a slight problem with performance of matsetvalues when inserting values row-by-row on single processor instead of column-by-column. > > ???? ierr = MatCreate(PETSC_COMM_WORLD,&L); > ???? ierr = MatSetSizes(L,PETSC_DECIDE,PETSC_DECIDE,n,n); > ???? ierr = MatSetFromOptions(L); > ???? MatSeqAIJSetPreallocation(L,PETSC_NULL,nnz); > > ? ? (...) ??? > > ???? MatSetValues(L,rows,indices_y,1,&col,values,INSERT_VALUES); > vs ??? > ???? MatSetValues(L,1,&row,cols,indices_x,values,INSERT_VALUES); > > First way goes quick and gives me mallocs in matsetvalues = 0 in -info > While the second one takes much longer and does mallocs. > > Why is it so that inserting rows is slower than inserting columns when proper nnz was provided? Or maybe there is my mistake here somewhere? The order or the way the values are inserted shouldn't matter. As long as the correct nozero structure is provided [with nnz] - there should be no additional mallocs. you can check with MatView() to see if the matrix is anyway different in the 2 cases. Satish > > Regards > From Juergen.Duerrwang at iosb.fraunhofer.de Thu May 26 06:43:56 2011 From: Juergen.Duerrwang at iosb.fraunhofer.de (=?iso-8859-1?Q?D=FCrrwang=2C_J=FCrgen?=) Date: Thu, 26 May 2011 13:43:56 +0200 Subject: [petsc-users] Matload failed with petsc-3.1-p8 Message-ID: Hi, I have some strange failure using petsc-3.1-p8. Before I used petsc-3.1-p3 and everything works fine. Now in this code block occurs a failure: /* Read in matrix */ ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD,"A.dat",FILE_MODE_READ,&viewer);CHKERRQ(ierr); ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr); ierr = MatLoad(A, viewer);CHKERRQ(ierr); //ierr = MatLoad(viewer,MATMPIAIJ,&A);CHKERRQ(ierr); this was the old call ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); and this is the error Message: >singleCore.bat [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Read from file failed! [0]PETSC ERROR: Read past end of file! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: HG Date: [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: petscFirstParallel on a arch-win3 named MRD-PC43 by duerrw Thu May 26 13:33:51 2011 [0]PETSC ERROR: Libraries linked from /cygdrive/c/projekte/common/petsc/petsc-dev/arch-win32/lib [0]PETSC ERROR: Configure run at Fri May 20 17:09:32 2011 [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc=0 --with-cxx="win32fe cl" --with-mpi-include=/cygdrive/c/Programme/Microsoft_HPC_Pack_2008_SDK/Include --with-mpi-lib=/cygdrive/c/Programme/Microsoft_HPC_Pack_2008_SDK/Lib/i386/msmpi.lib CXXFLAGS="-O2 -Ob2 -Oi -Ot -Oy -GT -GL -MD -openmp -FD -EHsc -Zp8 -W3 -Zi" CFLAGS="-O2 -Ob2 -Oi -Ot -Oy -GT -GL -MD -openmp -FD -EHsc -Zp8 -W3 -Zi" PETSC_ARCH=arch-win32 --download-c-blas-lapack=1 --with-c2html=0 --useThreads=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PetscBinaryRead() line 274 in src/sys/fileio/C:\projekte\common\petsc\PETSC-~1\src\sys\fileio\sysio.c [0]PETSC ERROR: MatLoad_SeqAIJ() line 3717 in src/mat/impls/aij/seq/C:\projekte\common\petsc\PETSC-~1\src\mat\impls\aij\seq\aij.c [0]PETSC ERROR: MatLoad() line 868 in src/mat/interface/C:\projekte\common\petsc\PETSC-~1\src\mat\INTERF~1\matrix.c [0]PETSC ERROR: User provided function() line 202 in "unknowndirectory/".\petscFirstpara.cpp Has somebody an idea? Regards J?rgen -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartlomiej.wach at yahoo.pl Thu May 26 06:46:46 2011 From: bartlomiej.wach at yahoo.pl (=?utf-8?B?QmFydMWCb21pZWogVw==?=) Date: Thu, 26 May 2011 12:46:46 +0100 (BST) Subject: [petsc-users] Insering rows vs inserting columns In-Reply-To: Message-ID: <366242.9423.qm@web28315.mail.ukl.yahoo.com> That helped. I am also wondering about SEQ and MPI matrices, do they differ more than just in way of allocating data? When I run a small problem on 2 processes on single machine, it slows down greatly if I use MatCreateMPIAIJ(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,n,n,5,PETSC_NULL,5,PETSC_NULL,&L); As I get mallocs during mat assembly while o_nz and d_nz are set to 5 where 5 is max nonzeros per whole row. I also get stash:0 assuring that values are set by correct process. Any hints on this one? Thank you Regards Bartek Wach The order or the way the values are inserted shouldn't matter. As long as the correct nozero structure is provided [with nnz] - there should be no additional mallocs. you can check with MatView() to see if the matrix is anyway different in the 2 cases. Satish -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Thu May 26 07:20:35 2011 From: jed at 59A2.org (Jed Brown) Date: Thu, 26 May 2011 14:20:35 +0200 Subject: [petsc-users] Insering rows vs inserting columns In-Reply-To: <366242.9423.qm@web28315.mail.ukl.yahoo.com> References: <366242.9423.qm@web28315.mail.ukl.yahoo.com> Message-ID: On Thu, May 26, 2011 at 13:46, Bart?omiej W wrote: > That helped. > > I am also wondering about SEQ and MPI matrices, do they differ more than > just in way of allocating data? When I run a small problem on 2 processes on > single machine, it slows down greatly if I use > > > MatCreateMPIAIJ(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,n,n,5,PETSC_NULL,5,PETSC_NULL,&L); > > As I get mallocs during mat assembly while o_nz and d_nz are set to 5 where > 5 is max nonzeros per whole row. I also get stash:0 assuring that values are > set by correct process. Your preallocation is either incorrect or it is getting lost. Try MAT_NEW_NONZERO_ALLOCATION_ERR http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Mat/MatSetOption.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 26 09:03:02 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 26 May 2011 09:03:02 -0500 Subject: [petsc-users] Matload failed with petsc-3.1-p8 In-Reply-To: References: Message-ID: 2011/5/26 D?rrwang, J?rgen > Hi, > > > > I have some strange failure using petsc-3.1-p8. > Before I used petsc-3.1-p3 and everything works fine. Now in this code block > occurs a failure: > 1) This appears to be petsc-dev, not petsc-3.1-p8 2) The MatLoad() format did not change. You can send the binary file to petsc-maint at mcs.anl.gov and we can look at it. Matt > > > > > /* Read in matrix */ > > ierr = > PetscViewerBinaryOpen(PETSC_COMM_WORLD,"A.dat",FILE_MODE_READ,&viewer);CHKERRQ(ierr); > > ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr); > > ierr = MatLoad(A, viewer);CHKERRQ(ierr); > //ierr > = MatLoad(viewer,MATMPIAIJ,&A);CHKERRQ(ierr); this was the old call > > ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); > > > > > > and this is the error Message: > > > > >singleCore.bat > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > > [0]PETSC ERROR: Read from file failed! > > [0]PETSC ERROR: Read past end of file! > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: Petsc Development HG revision: HG Date: > > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > [0]PETSC ERROR: See docs/index.html for manual pages. > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: petscFirstParallel on a arch-win3 named MRD-PC43 by duerrw > Thu May 26 13:33:51 2011 > > [0]PETSC ERROR: Libraries linked from > /cygdrive/c/projekte/common/petsc/petsc-dev/arch-win32/lib > > [0]PETSC ERROR: Configure run at Fri May 20 17:09:32 2011 > > [0]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-fc=0 > --with-cxx="win32fe cl" > --with-mpi-include=/cygdrive/c/Programme/Microsoft_HPC_Pack_2008_SDK/Include > --with-mpi-lib=/cygdrive/c/Programme/Microsoft_HPC_Pack_2008_SDK/Lib/i386/msmpi.lib > CXXFLAGS="-O2 -Ob2 -Oi -Ot -Oy -GT -GL -MD -openmp -FD -EHsc -Zp8 -W3 -Zi" > CFLAGS="-O2 -Ob2 -Oi -Ot -Oy -GT -GL -MD -openmp -FD -EHsc -Zp8 -W3 -Zi" > PETSC_ARCH=arch-win32 --download-c-blas-lapack=1 --with-c2html=0 > --useThreads=0 > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > > [0]PETSC ERROR: PetscBinaryRead() line 274 in > src/sys/fileio/C:\projekte\common\petsc\PETSC-~1\src\sys\fileio\sysio.c > > [0]PETSC ERROR: MatLoad_SeqAIJ() line 3717 in > src/mat/impls/aij/seq/C:\projekte\common\petsc\PETSC-~1\src\mat\impls\aij\seq\aij.c > > [0]PETSC ERROR: MatLoad() line 868 in > src/mat/interface/C:\projekte\common\petsc\PETSC-~1\src\mat\INTERF~1\matrix.c > > [0]PETSC ERROR: User provided function() line 202 in > "unknowndirectory/".\petscFirstpara.cpp > > > > Has somebody an idea? > > > > Regards J?rgen > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Sun May 29 02:36:15 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Sun, 29 May 2011 00:36:15 -0700 Subject: [petsc-users] PETSc with Qt Message-ID: HI guys, Quick question: Is there a way to compile/link PETSc library to a project being done in Qt? Thanks, Mohammad -------------- next part -------------- An HTML attachment was scrubbed... URL: From longmin.ran at gmail.com Mon May 30 07:51:44 2011 From: longmin.ran at gmail.com (Longmin RAN) Date: Mon, 30 May 2011 14:51:44 +0200 Subject: [petsc-users] what does it mean with "-1.#IND" in solution vector? Message-ID: Dear all, I'm using ksp with external superlu package to solve my linear systems. The system is sparse, real and symmetric, I'm expecting unique solution with normal values. However, the KSPSolve returned with error 0, but the norm of error is "1.#QNAN", and in the solution vector some values are "-1.#IND". What does this mean ? Any help is appreciated. Thanks. Cheers, Longmin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 30 18:29:06 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 30 May 2011 18:29:06 -0500 Subject: [petsc-users] PETSc with Qt In-Reply-To: References: Message-ID: On Sun, May 29, 2011 at 2:36 AM, Mohammad Mirzadeh wrote: > HI guys, > > Quick question: Is there a way to compile/link PETSc library to a project > being done in Qt? > Yes, just link in the libraries. For the current PETSc release, its just libpetsc. Matt > Thanks, > Mohammad > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 30 18:31:48 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 30 May 2011 18:31:48 -0500 Subject: [petsc-users] what does it mean with "-1.#IND" in solution vector? In-Reply-To: References: Message-ID: On Mon, May 30, 2011 at 7:51 AM, Longmin RAN wrote: > Dear all, > > I'm using ksp with external superlu package to solve my linear systems. The > system is sparse, real and symmetric, I'm expecting unique solution with > normal values. However, the KSPSolve returned with error 0, but the norm of > error is "1.#QNAN", and in the solution vector some values are "-1.#IND". > What does this mean ? Any help is appreciated. Thanks. > You generate a NaN somewhere. I usually check for this using norms, since NaN propagate. Check the norm of the rhs, initial solution, matrix, etc. Matt > Cheers, > > Longmin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon May 30 18:38:31 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 31 May 2011 01:38:31 +0200 Subject: [petsc-users] what does it mean with "-1.#IND" in solution vector? In-Reply-To: References: Message-ID: On Tue, May 31, 2011 at 01:31, Matthew Knepley wrote: > You generate a NaN somewhere. I usually check for this using norms, since > NaN propagate. Check the norm > of the rhs, initial solution, matrix, etc. > You can also run with -fp_trap, possibly in a debugger, and it should break at the first place the NaN was computed. This works better (on more platforms) with petsc-dev. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danesh.daroui at ltu.se Tue May 31 01:02:48 2011 From: danesh.daroui at ltu.se (Danesh Daroui) Date: Tue, 31 May 2011 08:02:48 +0200 Subject: [petsc-users] ILU preconditioner Message-ID: <1306821768.7496.3.camel@linux-sclt.site> Hi, I have a complex system of equations and I use PETSc to solve the system. I also use MKL to create the preconditioner using ILU. The ILU preconditioner is created only on the real part of the coefficient matrix and is stored in CSR format with the data type of double precision. I wanted to ask if it is possible to use this preconditioner and pass it to PETSc in CSR format? Can it be used with PETSc complex solver although the preconditioner is in double precision format? Regards, Danesh From longmin.ran at gmail.com Tue May 31 02:41:55 2011 From: longmin.ran at gmail.com (Longmin RAN) Date: Tue, 31 May 2011 09:41:55 +0200 Subject: [petsc-users] what does it mean with "-1.#IND" in solution vector? In-Reply-To: References: Message-ID: Thank you guys for the tips. Longmmin On Tue, May 31, 2011 at 1:38 AM, Jed Brown wrote: > On Tue, May 31, 2011 at 01:31, Matthew Knepley wrote: > >> You generate a NaN somewhere. I usually check for this using norms, since >> NaN propagate. Check the norm >> of the rhs, initial solution, matrix, etc. >> > > You can also run with -fp_trap, possibly in a debugger, and it should break > at the first place the NaN was computed. This works better (on more > platforms) with petsc-dev. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.witkowski at tu-dresden.de Tue May 31 02:44:34 2011 From: thomas.witkowski at tu-dresden.de (Thomas Witkowski) Date: Tue, 31 May 2011 09:44:34 +0200 Subject: [petsc-users] Memory allocated for a matrix Message-ID: <4DE49C62.90509@tu-dresden.de> Is there a simple way to get the amount of memory that was allocated for a specific matrix? Thomas From agrayver at gfz-potsdam.de Tue May 31 04:16:57 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Tue, 31 May 2011 11:16:57 +0200 Subject: [petsc-users] Different factorizations within one scope Message-ID: <4DE4B209.3020808@gfz-potsdam.de> Hello, Now I use MUMPS to solve Helmholtz equation for electromagnetic problem in frequency domain. My system matrix depends on frequency, since I have more than one frequency I need to do factorization for every one. Within one MPI communicator it is implemented as a loop over frequencies, where I assemble new system matrix at each iteration, factorize it and solve system for many RHS. The problem is that at each iteration previous factorization has been overwritten by new one, but now I came to point where I need factorizations for previous frequencies later in my program and that means that they need to be stored somewhere. My question is that possible to create several factorizations using PETSc classes and MUMPS package, store them and use later any you need? Regards, Alexander From knepley at gmail.com Tue May 31 08:09:21 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2011 08:09:21 -0500 Subject: [petsc-users] ILU preconditioner In-Reply-To: <1306821768.7496.3.camel@linux-sclt.site> References: <1306821768.7496.3.camel@linux-sclt.site> Message-ID: That would take knowledge of the internals of PETSc, since we use specialized data structures internally to speed up the triangular solve. You would have to look at the PETSc code. Thanks, Matt On Tue, May 31, 2011 at 1:02 AM, Danesh Daroui wrote: > > Hi, > > I have a complex system of equations and I use PETSc to solve the > system. I also use MKL to create the preconditioner using ILU. The ILU > preconditioner is created only on the real part of the coefficient > matrix and is stored in CSR format with the data type of double > precision. I wanted to ask if it is possible to use this preconditioner > and pass it to PETSc in CSR format? Can it be used with PETSc complex > solver although the preconditioner is in double precision format? > > Regards, > > Danesh > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 08:11:46 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2011 08:11:46 -0500 Subject: [petsc-users] Memory allocated for a matrix In-Reply-To: <4DE49C62.90509@tu-dresden.de> References: <4DE49C62.90509@tu-dresden.de> Message-ID: There is no simple way. There are possibly some complicated ways. Matt On Tue, May 31, 2011 at 2:44 AM, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > Is there a simple way to get the amount of memory that was allocated for a > specific matrix? > > Thomas > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 08:13:28 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2011 08:13:28 -0500 Subject: [petsc-users] Different factorizations within one scope In-Reply-To: <4DE4B209.3020808@gfz-potsdam.de> References: <4DE4B209.3020808@gfz-potsdam.de> Message-ID: On Tue, May 31, 2011 at 4:16 AM, Alexander Grayver wrote: > Hello, > > Now I use MUMPS to solve Helmholtz equation for electromagnetic problem in > frequency domain. My system matrix depends on frequency, since I have more > than one frequency I need to do factorization for every one. > Within one MPI communicator it is implemented as a loop over frequencies, > where I assemble new system matrix at each iteration, factorize it and solve > system for many RHS. The problem is that at each iteration previous > factorization has been overwritten by new one, but now I came to point where > I need factorizations for previous frequencies later in my program and that > means that they need to be stored somewhere. > > My question is that possible to create several factorizations using PETSc > classes and MUMPS package, store them and use later any you need? > The easiest way to do this is to create several KSP objects, which will have associated PC and Mat objects. You can resolve with each one later, and it will preserve the existing factorization. Matt > Regards, > Alexander > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Tue May 31 08:13:30 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 31 May 2011 15:13:30 +0200 Subject: [petsc-users] ILU preconditioner In-Reply-To: <1306821768.7496.3.camel@linux-sclt.site> References: <1306821768.7496.3.camel@linux-sclt.site> Message-ID: On Tue, May 31, 2011 at 08:02, Danesh Daroui wrote: > I have a complex system of equations and I use PETSc to solve the > system. I also use MKL to create the preconditioner using ILU. The ILU > preconditioner is created only on the real part of the coefficient > matrix and is stored in CSR format with the data type of double > precision. I wanted to ask if it is possible to use this preconditioner > and pass it to PETSc in CSR format? Can it be used with PETSc complex > solver although the preconditioner is in double precision format? > You can create a PCShell() that manages the factorization using MKL and applies the preconditioner stored in whatever format MKL likes. PETSc would not know that it is ILU or otherwise. It sounds like that would do what you want. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Tue May 31 08:21:43 2011 From: jed at 59A2.org (Jed Brown) Date: Tue, 31 May 2011 15:21:43 +0200 Subject: [petsc-users] Memory allocated for a matrix In-Reply-To: <4DE49C62.90509@tu-dresden.de> References: <4DE49C62.90509@tu-dresden.de> Message-ID: On Tue, May 31, 2011 at 09:44, Thomas Witkowski < thomas.witkowski at tu-dresden.de> wrote: > Is there a simple way to get the amount of memory that was allocated for a > specific matrix? If complied --with-log=1 (the default), then PetscLogDouble allocated = ((PetscObject)mat)->mem; will give you the total amount of memory that was logged as being allocated for the matrix. This is not necessarily everything, and I don't know if reallocation (if you didn't preallocate correctly) is logged consistently. -------------- next part -------------- An HTML attachment was scrubbed... URL: From renzhengyong at gmail.com Tue May 31 08:29:06 2011 From: renzhengyong at gmail.com (RenZhengYong) Date: Tue, 31 May 2011 15:29:06 +0200 Subject: [petsc-users] question on extracting a column from a matrix Message-ID: I have a linear system of equation like [A B] [E]= [C] [F G] [S]= [D], where size of matrix A is NxN, B is NxNs, F is Ns_N, G is Ns_Ns; size of vector E is N, S is Ns, C is N and D is N. Instead of solving them together, I prefer to solve it by: (1) AX=B, AY=C; (2) QS=P, Q= G-FX, P=D-FY, due to the dense property of G. In step (1), I need to extract a new "vec" Bi from "mat" B, according to the way of PETSc telling me. My question is that could I do it as follows, for a given sparse matrix B, first get transpose of B by "MatTranspose(B, MatReuse, B_t); then, form a new "vec" Bi by "MatGetRow(...)"? If answer is yes, is it expensive? -- Zhengyong Ren AUG Group, Institute of Geophysics Department of Geosciences, ETH Zurich NO H 47 Sonneggstrasse 5 CH-8092, Z?rich, Switzerland Tel: +41 44 633 37561 e-mail: zhengyong.ren at aug.ig.erdw.ethz.ch Gmail: renzhengyong at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Tue May 31 08:30:43 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Tue, 31 May 2011 15:30:43 +0200 Subject: [petsc-users] Different factorizations within one scope In-Reply-To: References: <4DE4B209.3020808@gfz-potsdam.de> Message-ID: <4DE4ED83.20706@gfz-potsdam.de> Thank you Matthew, that's what I wanted to hear. Regards, Alexander On 31.05.2011 15:13, Matthew Knepley wrote: > On Tue, May 31, 2011 at 4:16 AM, Alexander Grayver > > wrote: > > Hello, > > Now I use MUMPS to solve Helmholtz equation for electromagnetic > problem in frequency domain. My system matrix depends on > frequency, since I have more than one frequency I need to do > factorization for every one. > Within one MPI communicator it is implemented as a loop over > frequencies, where I assemble new system matrix at each iteration, > factorize it and solve system for many RHS. The problem is that at > each iteration previous factorization has been overwritten by new > one, but now I came to point where I need factorizations for > previous frequencies later in my program and that means that they > need to be stored somewhere. > > My question is that possible to create several factorizations > using PETSc classes and MUMPS package, store them and use later > any you need? > > > The easiest way to do this is to create several KSP objects, which > will have associated PC and Mat objects. You can resolve with each one > later, > and it will preserve the existing factorization. > > Matt > > Regards, > Alexander > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Thomas.Witkowski at tu-dresden.de Tue May 31 12:28:32 2011 From: Thomas.Witkowski at tu-dresden.de (Thomas Witkowski) Date: Tue, 31 May 2011 19:28:32 +0200 Subject: [petsc-users] Memory allocated for a matrix In-Reply-To: References: <4DE49C62.90509@tu-dresden.de> Message-ID: <20110531192832.rg0fkwatk4s8kc44@mail.zih.tu-dresden.de> And what would be a complicated way to get this info? Thomas Zitat von Matthew Knepley : > There is no simple way. There are possibly some complicated ways. > > Matt > > On Tue, May 31, 2011 at 2:44 AM, Thomas Witkowski < > thomas.witkowski at tu-dresden.de> wrote: > >> Is there a simple way to get the amount of memory that was allocated for a >> specific matrix? >> >> Thomas >> > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > From knepley at gmail.com Tue May 31 12:34:43 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2011 12:34:43 -0500 Subject: [petsc-users] Memory allocated for a matrix In-Reply-To: <20110531192832.rg0fkwatk4s8kc44@mail.zih.tu-dresden.de> References: <4DE49C62.90509@tu-dresden.de> <20110531192832.rg0fkwatk4s8kc44@mail.zih.tu-dresden.de> Message-ID: On Tue, May 31, 2011 at 12:28 PM, Thomas Witkowski < Thomas.Witkowski at tu-dresden.de> wrote: > And what would be a complicated way to get this info? > 1) Use PetscMallocGetCurrentUsage(PetscLogDouble *space) before creation and after setup without doing anything else in between 2) Walk the stack of all memory allocations, picking out those that have Mat functions in the stack Matt Thomas > > Zitat von Matthew Knepley : > > There is no simple way. There are possibly some complicated ways. >> >> Matt >> >> On Tue, May 31, 2011 at 2:44 AM, Thomas Witkowski < >> thomas.witkowski at tu-dresden.de> wrote: >> >> Is there a simple way to get the amount of memory that was allocated for >>> a >>> specific matrix? >>> >>> Thomas >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Tue May 31 14:41:55 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 31 May 2011 14:41:55 -0500 Subject: [petsc-users] question on extracting a column from a matrix In-Reply-To: References: Message-ID: This is virtually SCHUR complement. You may take a look at petsc-dev/src/ksp/ksp/examples/tests/ex11.c and petsc-dev/src/ksp/ksp/examples/tests/makefile: runex11 Hong On Tue, May 31, 2011 at 8:29 AM, RenZhengYong wrote: > I have a linear system of equation like > ?[A? B]? [E]= [C] > ?[F? G]? [S]= [D], > where size of matrix A is NxN, B is NxNs, F is Ns_N, G is Ns_Ns; size of > vector E is N, S is Ns, C is N and D is N. > Instead of solving them together, I prefer to solve it by: > (1) AX=B, AY=C; > (2) QS=P, Q= G-FX, P=D-FY, > due to the dense property of G. > > In step (1), I need to extract a new "vec" Bi from "mat" B, according to the > way of PETSc telling me. My question is that could I do it as follows, > for a given sparse matrix B, first get transpose of B by "MatTranspose(B, > MatReuse, B_t); then, form a new "vec" Bi by "MatGetRow(...)"? > If answer is yes, is it expensive? > > > > > -- > Zhengyong Ren > AUG Group, Institute of Geophysics > Department of Geosciences, ETH Zurich > NO H 47 Sonneggstrasse 5 > CH-8092, Z?rich, Switzerland > Tel: +41 44 633 37561 > e-mail: zhengyong.ren at aug.ig.erdw.ethz.ch > Gmail: renzhengyong at gmail.com > From bsmith at mcs.anl.gov Tue May 31 14:48:45 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 31 May 2011 14:48:45 -0500 Subject: [petsc-users] question on extracting a column from a matrix In-Reply-To: References: Message-ID: <97B4BA05-E924-4D6F-A316-2CDDC819267F@mcs.anl.gov> With petsc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html there are very easy ways to solve the Schur complement systems using PCFIELDSPLIT see http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/PC/PCFIELDSPLIT.html Barry On May 31, 2011, at 2:41 PM, Hong Zhang wrote: > This is virtually SCHUR complement. You may take a look at > petsc-dev/src/ksp/ksp/examples/tests/ex11.c and > petsc-dev/src/ksp/ksp/examples/tests/makefile: runex11 > > Hong > > On Tue, May 31, 2011 at 8:29 AM, RenZhengYong wrote: >> I have a linear system of equation like >> [A B] [E]= [C] >> [F G] [S]= [D], >> where size of matrix A is NxN, B is NxNs, F is Ns_N, G is Ns_Ns; size of >> vector E is N, S is Ns, C is N and D is N. >> Instead of solving them together, I prefer to solve it by: >> (1) AX=B, AY=C; >> (2) QS=P, Q= G-FX, P=D-FY, >> due to the dense property of G. >> >> In step (1), I need to extract a new "vec" Bi from "mat" B, according to the >> way of PETSc telling me. My question is that could I do it as follows, >> for a given sparse matrix B, first get transpose of B by "MatTranspose(B, >> MatReuse, B_t); then, form a new "vec" Bi by "MatGetRow(...)"? >> If answer is yes, is it expensive? >> >> >> >> >> -- >> Zhengyong Ren >> AUG Group, Institute of Geophysics >> Department of Geosciences, ETH Zurich >> NO H 47 Sonneggstrasse 5 >> CH-8092, Z?rich, Switzerland >> Tel: +41 44 633 37561 >> e-mail: zhengyong.ren at aug.ig.erdw.ethz.ch >> Gmail: renzhengyong at gmail.com >> From mirzadeh at gmail.com Tue May 31 16:04:37 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Tue, 31 May 2011 14:04:37 -0700 Subject: [petsc-users] PETSc with Qt In-Reply-To: References: Message-ID: Thanks Matt. I actually did that with -lpetsc but it complains that cannot find some functions related to mpi ... I'm thinking(actually right now) this could be since I have to configure Qt to run mpirun, correct? On Mon, May 30, 2011 at 4:29 PM, Matthew Knepley wrote: > On Sun, May 29, 2011 at 2:36 AM, Mohammad Mirzadeh wrote: > >> HI guys, >> >> Quick question: Is there a way to compile/link PETSc library to a project >> being done in Qt? >> > > Yes, just link in the libraries. For the current PETSc release, its just > libpetsc. > > Matt > > >> Thanks, >> Mohammad >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 16:06:32 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2011 16:06:32 -0500 Subject: [petsc-users] PETSc with Qt In-Reply-To: References: Message-ID: On Tue, May 31, 2011 at 4:04 PM, Mohammad Mirzadeh wrote: > Thanks Matt. I actually did that with -lpetsc but it complains that cannot > find some functions related to mpi ... I'm thinking(actually right now) this > could be since I have to configure Qt to run mpirun, correct? > No, you need the whole PETSc link line (as we say in the manual or installation instructions). You can get it using make getlinklibs Matt > On Mon, May 30, 2011 at 4:29 PM, Matthew Knepley wrote: > >> On Sun, May 29, 2011 at 2:36 AM, Mohammad Mirzadeh wrote: >> >>> HI guys, >>> >>> Quick question: Is there a way to compile/link PETSc library to a project >>> being done in Qt? >>> >> >> Yes, just link in the libraries. For the current PETSc release, its just >> libpetsc. >> >> Matt >> >> >>> Thanks, >>> Mohammad >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirzadeh at gmail.com Tue May 31 16:46:29 2011 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Tue, 31 May 2011 14:46:29 -0700 Subject: [petsc-users] PETSc with Qt In-Reply-To: References: Message-ID: Matt, Thank you so much. Its resolved now On Tue, May 31, 2011 at 2:06 PM, Matthew Knepley wrote: > On Tue, May 31, 2011 at 4:04 PM, Mohammad Mirzadeh wrote: > >> Thanks Matt. I actually did that with -lpetsc but it complains that cannot >> find some functions related to mpi ... I'm thinking(actually right now) this >> could be since I have to configure Qt to run mpirun, correct? >> > > No, you need the whole PETSc link line (as we say in the manual or > installation instructions). You can get it using > > make getlinklibs > > Matt > > >> On Mon, May 30, 2011 at 4:29 PM, Matthew Knepley wrote: >> >>> On Sun, May 29, 2011 at 2:36 AM, Mohammad Mirzadeh wrote: >>> >>>> HI guys, >>>> >>>> Quick question: Is there a way to compile/link PETSc library to a >>>> project being done in Qt? >>>> >>> >>> Yes, just link in the libraries. For the current PETSc release, its just >>> libpetsc. >>> >>> Matt >>> >>> >>>> Thanks, >>>> Mohammad >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: