<div dir="ltr"><div><div><div><div><div><div>Barry,<br></div><div><br>I am trying to reproduce this issue using a pure PETSc code. VecLoad does not work for me. I do not know why. Anyway, I can reproduce this using a very small system. Here are some info:<br><br></div>Mat, A<br>Mat Object:() 2 MPI processes<br> type: mpiaij<br>row 0: (0, 1.) <br>row 1: (0, -0.820827) (1, 1.51669) (2, -0.820827) <br>row 2: (1, -0.820827) (2, 1.51669) (3, -0.820827) <br>row 3: (2, -0.820827) (3, 1.51669) (4, -0.820827) <br>row 4: (3, -0.820827) (4, 1.51669) (5, -0.820827) <br>row 5: (4, -0.820827) (5, 1.51669) (6, -0.820827) <br>row 6: (5, -0.820827) (6, 1.51669) (7, -0.820827) <br>row 7: (6, -0.820827) (7, 1.51669) (8, -0.820827) <br>row 8: (8, 1.) <br><br><br></div>Right hand side b:<br><br>Vec Object: 2 MPI processes<br> type: mpi<br>Process [0]<br>0.<br>-0.356693<br>-0.50444<br>-0.356693<br>-5.55112e-17<br>Process [1]<br>0.356693<br>0.50444<br>0.356693<br>0.<br><br><br></div>Mat Null space N(A):<br><br>Vec Object: 2 MPI processes<br> type: mpi<br>Process [0]<br>0.<br>0.191342<br>0.353553<br>0.46194<br>0.5<br>Process [1]<br>0.46194<br>0.353553<br>0.191342<br>6.12323e-17<br><br><br></div>Please run with two MPI threads using -ksp_pc_side <span style="background-color:rgb(255,0,0)">right</span> -pc_type bjacobi and -ksp_pc_side <span style="background-color:rgb(255,0,0)">left</span> -pc_type bjacobi. Will produce different solutions. The one obtained with using "left" is correct (we have an analytical solution).<br><br></div>I also attached data for matrix, rhs and nullspace, but I am not sure if you can read them or not. I can load mat.dat, but I could not read rhs.dat and nullspace.dat.<br><br></div>Fande,<br><div><div><div><div><br><br></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 11, 2016 at 3:44 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Fande,<br>
<br>
Could you send me (<a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a>) a non symmetric matrix you have that has a different null space for A and A'. This would be one that is failing with right preconditioning. Smaller the better but whatever size you have. Run the code with -ksp_view_mat binary and send the resulting file called binaryoutput.<br>
<br>
I need a test matrix to update the PETSc code for this case.<br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
Barry<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> On Oct 11, 2016, at 3:04 PM, Kong, Fande <<a href="mailto:fande.kong@inl.gov">fande.kong@inl.gov</a>> wrote:<br>
><br>
><br>
><br>
> On Tue, Oct 11, 2016 at 12:18 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> > On Oct 11, 2016, at 12:01 PM, Kong, Fande <<a href="mailto:fande.kong@inl.gov">fande.kong@inl.gov</a>> wrote:<br>
> ><br>
> ><br>
> ><br>
> > On Tue, Oct 11, 2016 at 10:39 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> ><br>
> > > On Oct 11, 2016, at 9:33 AM, Kong, Fande <<a href="mailto:fande.kong@inl.gov">fande.kong@inl.gov</a>> wrote:<br>
> > ><br>
> > > Barry, Thanks so much for your explanation. It helps me a lot.<br>
> > ><br>
> > > On Mon, Oct 10, 2016 at 4:00 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> > ><br>
> > > > On Oct 10, 2016, at 4:01 PM, Kong, Fande <<a href="mailto:fande.kong@inl.gov">fande.kong@inl.gov</a>> wrote:<br>
> > > ><br>
> > > > Hi All,<br>
> > > ><br>
> > > > I know how to remove the null spaces from a singular system using creating a MatNullSpace and attaching it to Mat.<br>
> > > ><br>
> > > > I was really wondering what is the philosophy behind this? The exact algorithms we are using in PETSc right now? Where we are dealing with this, preconditioner, linear solver, or nonlinear solver?<br>
> > ><br>
> > > It is in the Krylov solver.<br>
> > ><br>
> > > The idea is very simple. Say you have a singular A with null space N (that all values Ny are in the null space of A. So N is tall and skinny) and you want to solve A x = b where b is in the range of A. This problem has an infinite number of solutions Ny + x* since A (Ny + x*) = ANy + Ax* = Ax* = b where x* is the "minimum norm solution; that is Ax* = b and x* has the smallest norm of all solutions.<br>
> > ><br>
> > > With left preconditioning B A x = B b GMRES, for example, normally computes the solution in the as alpha_1 Bb + alpha_2 BABb + alpha_3 BABABAb + .... but the B operator will likely introduce some component into the direction of the null space so as GMRES continues the "solution" computed will grow larger and larger with a large component in the null space of A. Hence we simply modify GMRES a tiny bit by building the solution from alpha_1 (I-N)Bb + alpha_2 (I-N)BABb + alpha_3<br>
> > ><br>
> > > Does "I" mean an identity matrix? Could you possibly send me a link for this GMRES implementation, that is, how PETSc does this in the actual code?<br>
> ><br>
> > Yes.<br>
> ><br>
> > It is in the helper routine KSP_PCApplyBAorAB()<br>
> > #undef __FUNCT__<br>
> > #define __FUNCT__ "KSP_PCApplyBAorAB"<br>
> > PETSC_STATIC_INLINE PetscErrorCode KSP_PCApplyBAorAB(KSP ksp,Vec x,Vec y,Vec w)<br>
> > {<br>
> > PetscErrorCode ierr;<br>
> > PetscFunctionBegin;<br>
> > if (!ksp->transpose_solve) {<br>
> > ierr = PCApplyBAorAB(ksp->pc,ksp->pc_<wbr>side,x,y,w);CHKERRQ(ierr);<br>
> > ierr = KSP_RemoveNullSpace(ksp,y);<wbr>CHKERRQ(ierr);<br>
> > } else {<br>
> > ierr = PCApplyBAorABTranspose(ksp-><wbr>pc,ksp->pc_side,x,y,w);<wbr>CHKERRQ(ierr);<br>
> > }<br>
> > PetscFunctionReturn(0);<br>
> > }<br>
> ><br>
> ><br>
> > PETSC_STATIC_INLINE PetscErrorCode KSP_RemoveNullSpace(KSP ksp,Vec y)<br>
> > {<br>
> > PetscErrorCode ierr;<br>
> > PetscFunctionBegin;<br>
> > if (ksp->pc_side == PC_LEFT) {<br>
> > Mat A;<br>
> > MatNullSpace nullsp;<br>
> > ierr = PCGetOperators(ksp->pc,&A,<wbr>NULL);CHKERRQ(ierr);<br>
> > ierr = MatGetNullSpace(A,&nullsp);<wbr>CHKERRQ(ierr);<br>
> > if (nullsp) {<br>
> > ierr = MatNullSpaceRemove(nullsp,y);<wbr>CHKERRQ(ierr);<br>
> > }<br>
> > }<br>
> > PetscFunctionReturn(0);<br>
> > }<br>
> ><br>
> > "ksp->pc_side == PC_LEFT" deals with the left preconditioning Krylov methods only? How about the right preconditioning ones? Are they just magically right for the right preconditioning Krylov methods?<br>
><br>
> This is a good question. I am working on a branch now where I will add some more comprehensive testing of the various cases and fix anything that comes up.<br>
><br>
> Were you having trouble with ASM and bjacobi only for right preconditioning?<br>
><br>
><br>
> Yes. ASM and bjacobi works fine for left preconditioning NOT for RIGHT preconditioning. bjacobi converges, but produces a wrong solution. ASM needs more iterations, however the solution is right.<br>
><br>
><br>
><br>
> Note that when A is symmetric the range of A is orthogonal to null space of A so yes I think in that case it is just "magically right" but if A is not symmetric then I don't think it is "magically right". I'll work on it.<br>
><br>
><br>
> Barry<br>
><br>
> ><br>
> > Fande Kong,<br>
> ><br>
> ><br>
> > There is no code directly in the GMRES or other methods.<br>
> ><br>
> > ><br>
> > > (I-N)BABABAb + .... that is we remove from each new direction anything in the direction of the null space. Hence the null space doesn't directly appear in the preconditioner, just in the KSP method. If you attach a null space to the matrix, the KSP just automatically uses it to do the removal above.<br>
> > ><br>
> > > With right preconditioning the solution is built from alpha_1 b + alpha_2 ABb + alpha_3 ABABb + .... and again we apply (I-N) to each term to remove any part that is in the null space of A.<br>
> > ><br>
> > > Now consider the case A y = b where b is NOT in the range of A. So the problem has no "true" solution, but one can find a least squares solution by rewriting b = b_par + b_perp where b_par is in the range of A and b_perp is orthogonal to the range of A and solve instead A x = b_perp. If you provide a MatSetTransposeNullSpace() then KSP automatically uses it to remove b_perp from the right hand side before starting the KSP iterations.<br>
> > ><br>
> > > The manual pages for MatNullSpaceAttach() and MatTranposeNullSpaceAttach() discuss this an explain how it relates to the fundamental theorem of linear algebra.<br>
> > ><br>
> > > Note that for symmetric matrices the two null spaces are the same.<br>
> > ><br>
> > > Barry<br>
> > ><br>
> > ><br>
> > > A different note: This "trick" is not a "cure all" for a totally inappropriate preconditioner. For example if one uses for a preconditioner a direct (sparse or dense) solver or an ILU(k) one can end up with a very bad solver because the direct solver will likely produce a very small pivot at some point thus the triangular solver applied in the precondition can produce HUGE changes in the solution (that are not physical) and so the preconditioner basically produces garbage. On the other hand sometimes it works out ok.<br>
> > ><br>
> > > What preconditioners are appropriate? asm, bjacobi, amg? I have an example which shows lu and ilu indeed work, but asm and bjacobi do not at all. That is why I am asking questions about algorithms. I am trying to figure out a default preconditioner for several singular systems.<br>
> ><br>
> > Hmm, normally asm and bjacobi would be fine with this unless one or more of the subblocks are themselves singular (which normally won't happen). AMG can also work find sometimes.<br>
> ><br>
> > Can you send a sample code?<br>
> ><br>
> > Barry<br>
> ><br>
> > ><br>
> > > Thanks again.<br>
> > ><br>
> > ><br>
> > > Fande Kong,<br>
> > ><br>
> > ><br>
> > ><br>
> > > ><br>
> > > ><br>
> > > > Fande Kong,<br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>