[petsc-dev] Use KSPSetUpNorms_Private() in KSPSetFromOptions()

[Barry Smith]

On Feb 26, 2013, at 9:54 PM, Dmitry Karpeev <karpeev at mcs.anl.gov> wrote:

> Here's what I get with this simple example (source and makefile also attached):
> 
> 
> #include <petscksp.h>
> 
> #undef __FUNCT__
> #define __FUNCT__ "main"
> int main(int argc,char **args)
> {
>   KSP            ksp;
>   PetscErrorCode ierr;
>   PetscInitialize(&argc,&args,(char*)0,(char*)0);
>   ierr = KSPCreate(PETSC_COMM_WORLD, &ksp);CHKERRQ(ierr);
>   ierr = KSPSetPCSide(ksp, PC_RIGHT);CHKERRQ(ierr);
>   ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr);
>   ierr = KSPDestroy(&ksp);CHKERRQ(ierr);
>   PetscFinalize();
> }
> 
> 
> Then
> 
> [hal3 Tue Feb 26 09:34 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp >./ex -ksp_type preonly -ksp_pc_side left
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> [0]PETSC ERROR: No support for this operation for this object type!
> [0]PETSC ERROR: KSP preonly does not support RIGHT!
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development HG revision: 55699a42ce9360fb4641931ab7ec242cba302c58  HG Date: Mon Feb 25 22:15:44 2013 -0500
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: ./ex on a arch-gxx-real named hal3 by karpeev Tue Feb 26 21:37:13 2013
> [0]PETSC ERROR: Libraries linked from /home/karpeev/petsc/dev/c/arch-gxx-real/lib
> [0]PETSC ERROR: Configure run at Tue Feb 26 17:50:35 2013
> [0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gxx-real --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-clanguage=C++ --with-cxx=g++ --with-dynamic-loading=1 --with-python=1 --with-scalar-type=real --with-shared-libraries=1
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: KSPSetUpNorms_Private() line 385 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcreate.c
> [0]PETSC ERROR: KSPSetFromOptions() line 409 in /home/karpeev/petsc/dev/c/src/ksp/ksp/interface/itcl.c
> [0]PETSC ERROR: main() line 13 in ex.c
> application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0
> [unset]: aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0
> [hal3 Tue Feb 26 09:37 PM] ~/petsc/dev/c/src/ksp/ksp/examples/tutorials/tmp >
> 
> The problem seems to be this clause in KSPSetUpNorms_Private():
> 
> 
>         if (ksp->normtype == KSP_NORM_DEFAULT && i == KSP_NORM_NONE && ksp->normsupporttable[i][j] <= 1) {
>           continue; /* Skip because we don't want to default to no norms unless set by the KSP (preonly). */
> 
> 
> It is hit because of the code in KSPCreate_PREONLY() you quoted:
> 
>   ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); /* LEFT/RIGHT is arbitrary, so "support" both */
>   ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr);
> 
> The problem is that PC_RIGHT is preferred by KSP_NORM only with priority 1, which is too weak to overcome
> that "continue" clause above.

   Ok, this is a bug, we'll leave it to Jed to fix properly since he created this Frankensteinen monster.

> 
> One way to fix it is to jack up the priority of PC_RIGHT, but I still don't understand why this combination
> of norms and pc sides is being figured out in KSPSetFromOptions() (twice) with only partial KSP data.

   Because it want's to give as reasonable a possible solution to the user before the user gets to override with something else.

> And then again, this time "for real", in KSPSetUp().
> 
> Dmitry.
> 
> On Wed, Feb 27, 2013 at 2:52 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> On Feb 26, 2013, at 4:44 PM, Dmitry Karpeev <karpeev at mcs.anl.gov> wrote:
> 
> > These calls compute default values for normtype and pcside based on partial KSP options,
> > which can cause an unwarranted error to be thrown.
> > For example, in code there may be
> >
> > KSPSetPCSide(ksp, PC_RIGHT); /* hardwired default, e.g., in Moose. */
> >
> > Now, when I'm using command-line arguments
> > -ksp_type preonly
> > an error is thrown, since KSPPREONLY doesn't work with PC_RIGHT.
> 
>   Huh?
> 
> EXTERN_C_BEGIN
> #undef __FUNCT__
> #define __FUNCT__ "KSPCreate_PREONLY"
> PetscErrorCode  KSPCreate_PREONLY(KSP ksp)
> {
>   PetscErrorCode ierr;
> 
>   PetscFunctionBegin;
>   ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_LEFT,2);CHKERRQ(ierr); /* LEFT/RIGHT is arbitrary, so "support" both */
>   ierr = KSPSetSupportedNorm(ksp,KSP_NORM_NONE,PC_RIGHT,1);CHKERRQ(ierr);
> 
> Is it because MOOSE also hardwires the norm type and preonly doesn't support it? What is the full and complete error message.
> 
> > Fine, I try
> > -ksp_type preonly -ksp_pc_side left
> > but still get the same error, since KSPSetUpNorms_Private() is called after KSP type has been set to KSPPREONLY,
> > but BEFORE there was a chance to set the PC side!
> >
> > Is there any reason to keep these calls to KSPSetUpNorms_Private() in KSPSetFromOptions()?
> 
>    I think they are there so that users can overwrite them in the rest of KSPSetFromOptions(). If you move them to KSPSetUp() then the user cannot reset them.
> 
>    Barry
> 
> 
> > KSPSetUpNorms_Private() will be just about the first thing called in KSPSetUp() and will compute the appropriate normtype and pcside,
> > but now based on complete KSP options.
> >
> > Not to mention that calling XXXSetUp_YYY() XXXSetFromOptions() seems generally contrary to PETSc's design
> > philosophy.
> >
> > Dmitry.
> >
> >
> 
> 
> 
> 
> <ex.c><makefile>