[petsc-users] Use block Jacobi preconditioner with SNES

[Smith, Barry F.]

> On Aug 27, 2018, at 4:04 PM, Jed Brown <jed at jedbrown.org> wrote:
> 
> "Smith, Barry F." <bsmith at mcs.anl.gov> writes:
> 
>>    I have added new functionality within SNES to allow one to do as you desire in the branch barry/feature-snessetkspsetupcallback. Please see src/snes/examples/tests/ex3.c 
> 
> Note that this example is block Jacobi with O(1) sparse blocks per
> process, not variable-sized point-block Jacobi which I think is what Ali
> had in mind.

    It is almost a cut and paste of the original code provided, just set up as a callback. If it performs well convergence wise then one could consider implementing the (presumably) more efficient variable blocksize point block Jacobi (either as an entirely new preconditioner or as an option to the current point block Jacobi; I would advocate a new one PCVPBJACOBI in order to limit code complexity even though there might be a bit of code duplication.)



> 
>>    Please let us know if it works for you and if you have any questions or problems.
>> 
>>    Barry
>> 
>>   Note that it has to be handled by a callback called from within the SNES solver because that is the first time the matrix exists in a form that the block sizes may be set.
>> 
>>    Sorry for the runaround with so many emails. 
>> 
>> 
>>> On Aug 24, 2018, at 3:48 PM, Ali Reza Khaz'ali <arkhazali at cc.iut.ac.ir> wrote:
>>> 
>>> Hi,
>>> 
>>> I am trying to use block Jacobi preconditioner in SNES (SNESNEWTONLS). However, PCBJacobiGetSubKSP function returns an error stating "Object is in wrong state, Must call KSPSetUp() or PCSetUp() first". When I add KSPSetUp, I got and error from them as: "Must call DMShellSetGlobalVector() or DMShellSetCreateGlobalVector()", and if PCSetUp is added, "Object is in wrong state, Matrix must be set first" error is printed.
>>> 
>>> Below is a part of my code. It is run serially. Any help is much appreciated.
>>> 
>>>    ierr = SNESGetKSP(snes, &Petsc_ksp);
>>>    CHKERRQ(ierr);
>>>    ierr = KSPGetPC(Petsc_ksp, &Petsc_pc);
>>>    CHKERRQ(ierr);
>>>    ierr = KSPSetTolerances(Petsc_ksp, 1.e-3, 1e-3, PETSC_DEFAULT, 20000);
>>>    CHKERRQ(ierr);
>>>    ierr = SNESSetTolerances(snes, 1e-1, 1e-1, 1e-1, 2000, 2000);
>>>    CHKERRQ(ierr);
>>>    ierr = SNESSetType(snes, SNESNEWTONLS);
>>>    CHKERRQ(ierr);
>>>    ierr = KSPSetType(Petsc_ksp, KSPGMRES);
>>>    CHKERRQ(ierr);
>>>    ierr = PCSetType(Petsc_pc, PCBJACOBI);
>>>    CHKERRQ(ierr);
>>>    ierr = PCSetType(Petsc_pc, PCBJACOBI);
>>>    CHKERRQ(ierr);
>>>    ierr = PCBJacobiSetTotalBlocks(Petsc_pc, 2*Nx*Ny*Nz, SadeqSize);
>>>    CHKERRQ(ierr);
>>> 
>>>    SNESSetUp(snes);
>>>    CHKERRQ(ierr);
>>>    ierr = PCBJacobiGetSubKSP(Petsc_pc, &nLocal, &Firstly, &subKSP);
>>>    CHKERRQ(ierr);
>>> 
>>>    for (i = 0; i < nLocal; i++) {
>>>        ierr = KSPGetPC(subKSP[i], &SubPc);
>>>        CHKERRQ(ierr);
>>>        ierr = PCSetType(SubPc, PCLU);
>>>        CHKERRQ(ierr);
>>>        ierr = PCFactorSetMatSolverPackage(SubPc, "mkl_pardiso");
>>>        CHKERRQ(ierr);
>>>        ierr = KSPSetType(subKSP[i], KSPPREONLY);
>>>        CHKERRQ(ierr);
>>>        ierr = KSPSetTolerances(subKSP[i], 1.e-6, PETSC_DEFAULT, PETSC_DEFAULT, PETSC_DEFAULT);
>>>        CHKERRQ(ierr);
>>>    }
>>>    ierr = SNESSolve(snes, NULL, Petsc_X);
>>>    CHKERRQ(ierr);
>>> 
>>> 
>>> 
>>> -- 
>>> Ali Reza Khaz’ali
>>> Assistant Professor of Petroleum Engineering,
>>> Department of Chemical Engineering
>>> Isfahan University of Technology
>>> Isfahan, Iran
>>>