<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Again,<div><br></div><div>Thanks for everybody's prompt help. I got petsc3.2-p7 running again and that works so at least I am able to get results again. </div><div><br></div><div>I ran my test case with petsc3.2-p7 and petsc3.3-p5 and turned on ksp_view output. The file "output" is identical except for one line:</div><div><br></div><div>tolerance for zero pivot 2.22045e-14 for petsc3.3-p5 and</div><div>tolerance for zero pivot 1e-12 for petsc3.2-p7</div><div><br></div><div>This is the file "output" from ksp_view</div><div><br></div><div><div>KSP Object: 2 MPI processes</div><div> type: preonly</div><div> maximum iterations=10000, initial guess is zero</div><div> tolerances: relative=1e-05, absolute=1e-50, divergence=10000</div><div> left preconditioning</div><div> using NONE norm type for convergence test</div><div>PC Object: 2 MPI processes</div><div> type: lu</div><div> LU: out-of-place factorization</div><div> tolerance for zero pivot 2.22045e-14</div><div> matrix ordering: natural</div><div> factor fill ratio given 0, needed 0</div><div> Factored matrix follows:</div><div> Matrix Object: 2 MPI processes</div><div> type: mpiaij</div><div> rows=6957, cols=6957</div><div> package used to perform factorization: superlu_dist</div><div> total: nonzeros=0, allocated nonzeros=0</div><div> total number of mallocs used during MatSetValues calls =0</div><div> SuperLU_DIST run parameters:</div><div> Process grid nprow 2 x npcol 1 </div><div> Equilibrate matrix TRUE </div><div> Matrix input mode 1 </div><div> Replace tiny pivots TRUE </div><div> Use iterative refinement FALSE </div><div> Processors in row 2 col partition 1 </div><div> Row permutation LargeDiag </div><div> Column permutation METIS_AT_PLUS_A</div><div> Parallel symbolic factorization FALSE </div><div> Repeated factorization SamePattern_SameRowPerm</div><div> linear system matrix = precond matrix:</div><div> Matrix Object: 2 MPI processes</div><div> type: mpiaij</div><div> rows=6957, cols=6957</div><div> total: nonzeros=611043, allocated nonzeros=0</div><div> total number of mallocs used during MatSetValues calls =0</div><div> using I-node (on process 0) routines: found 1407 nodes, limit used is 5</div></div><div><br></div><div>The residual vector going into the KSP inversion is the same, but the inversion gives a different answer</div><div><br></div><div>petsc3.3-p5:</div><div><br></div><div><div>jacobian made 4.628e-01 seconds</div><div>matrix inverted 9.669e-01 seconds</div><div># iterations 1 residual0 1.824e-05 du 4.290e-05 solve time: 1.821e-02 seconds</div></div><div><br></div><div>petsc3.2-p7:</div><div><br></div><div><div>jacobian made 4.279e-01 seconds</div><div>matrix inverted 6.854e-01 seconds</div><div># iterations 1 residual0 1.824e-05 du 1.885e-05 solve time: 1.284e-02 seconds</div></div><div><br></div><div>Where the output is calculated as:</div><div><br></div><div><div style="margin: 0px; font-size: 11px; font-family: Menlo; "><div style="margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span><span style="color: #bb2ca2">double</span> resmax;</div><div style="margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span><span style="color: #3d1d81">VecNorm</span>(<span style="color: #4f8187">petsc_f</span>, <span style="color: #3d1d81">NORM_2</span>, &resmax );</div><p style="margin: 0px; min-height: 13px; "><span class="Apple-tab-span" style="white-space:pre"> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; color: rgb(61, 29, 129); "><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span></span>PetscGetTime<span style="color: #000000">(&time1);</span></div><div style="margin: 0px; color: rgb(79, 129, 135); "><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span>err = </span><span style="color: #3d1d81">KSPSolve</span><span style="color: #000000">(</span>ksp<span style="color: #000000">,</span>petsc_f<span style="color: #000000">,</span>petsc_du<span style="color: #000000">);</span></div><div style="margin: 0px; color: rgb(120, 73, 42); "><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span></span>CHKERRABORT<span style="color: #000000">(</span>MPI_COMM_WORLD<span style="color: #000000">,err);</span></div><p style="margin: 0px; min-height: 13px; "><span class="Apple-tab-span" style="white-space:pre"> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span><span style="color: #bb2ca2">double</span> resmax2;</div><div style="margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span><span style="color: #3d1d81">VecNorm</span>(<span style="color: #4f8187">petsc_du</span>, <span style="color: #3d1d81">NORM_2</span>, &resmax2 );</div><p style="margin: 0px; min-height: 13px; "><span class="Apple-tab-span" style="white-space:pre"> </span><br class="webkit-block-placeholder"></p><div style="margin: 0px; color: rgb(61, 29, 129); "><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span></span>KSPGetIterationNumber<span style="color: #000000">(</span><span style="color: #4f8187">ksp</span><span style="color: #000000">,&its);</span></div><div style="margin: 0px; color: rgb(61, 29, 129); "><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span></span>PetscGetTime<span style="color: #000000">(&time2);</span></div><div style="margin: 0px; "><span class="Apple-tab-span" style="white-space:pre"> </span>*<span style="color: #4f8187">gbl</span>-><span style="color: #703daa">log</span> << <span style="color: #d12f1b">"# iterations "</span> << its << <span style="color: #d12f1b">" residual0 "</span> << resmax << <span style="color: #d12f1b">" du "</span> << resmax2 << <span style="color: #d12f1b">" solve time: "</span> << time2-time1 << <span style="color: #d12f1b">" seconds"</span> << <span style="color: #3d1d81">endl</span>;</div><div><br></div></div></div><div style="margin: 0px; font-size: 11px; font-family: Menlo; "><br></div><div style="margin: 0px; font-size: 11px; font-family: Menlo; "><div style="font-family: Helvetica; font-size: medium; ">I can output the jacobian and make sure that is the same, but I'm guessing that it is. Any other suggestions of things to try? I'd be surprised if the zero-pivot thing had much to do with it although I could be wrong. I've arranged the matrices to try and make them diagonally dominant going in. Does anyone know how to change that setting of the top of their head (i.e. without me reading the manual).</div><div style="font-family: Helvetica; font-size: medium; "><br></div><div style="font-family: Helvetica; font-size: medium; ">Thanks again,</div><div style="font-family: Helvetica; font-size: medium; "><br></div><div style="font-family: Helvetica; font-size: medium; ">Brian</div><div style="font-family: Helvetica; font-size: medium; "><br></div><div style="font-family: Helvetica; font-size: medium; "><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div style="margin: 0px; font-size: 11px; font-family: Menlo; "><br></div></body></html>