[petsc-users] Trying to set up a field-split preconditioner

Sat Jul 20 16:57:51 CDT 2013

Hi Dmitry, I am using petsc 3.4.2  . I should have been clearer about the problem that I had. I was unable to see a call to  DMCreateFieldDecomposition  in gdb when I ran the code with a field split preconditioner specified. If the routine has not yet been updated, I can understand why it is causing problems.Thanks, 
Subramanya  

From: karpeev at mcs.anl.gov
Date: Sat, 20 Jul 2013 16:42:50 -0500
Subject: Fwd: [petsc-users] Trying to set up a field-split preconditioner
To: potaman at outlook.com; libmesh-users at lists.sourceforge.net

What version of petsc are you using?DMlibMesh needs to be updated to serve the splits via DMCreateFieldDecomposition() correctly to petsc-3.4. Note that DMCreateFieldDecomposition() is called by PCFieldSplit from PETSc, so you wouldn't see those calls in the libMesh source.

Dmitry.

On Sat, Jul 20, 2013 at 6:35 AM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi Matt, 

Libmesh does create the DM. This is the relevant code. 

ierr = DMCreateLibMesh(libMesh::COMM_WORLD, this->system(), &dm);CHKERRABORT(libMesh::COMM_WORLD, ierr);

ierr = DMSetFromOptions(dm); CHKERRABORT(libMesh::COMM_WORLD, ierr);

ierr = DMSetUp(dm); CHKERRABORT(libMesh::COMM_WORLD, ierr);

ierr = SNESSetDM(this->_snes, dm); CHKERRABORT(libMesh::COMM_WORLD, ierr);

I am unable to tell from the code whether the DMCreateFieldDecomposition is run at all. I will see if running in the debugger helps me figure it out.

Thanks,

Subramanya 

Date: Sat, 20 Jul 2013 06:22:30 -0500

Subject: Re: [petsc-users] Trying to set up a field-split preconditioner

From: knepley at gmail.com

To: potaman at outlook.com

CC: petsc-users at mcs.anl.gov

On Sat, Jul 20, 2013 at 6:19 AM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi Matt, 

The DM is created by the LibMesh code. The only thing I do directly with Petsc is set the solver prefixes which libmesh doesn't have an interface for at present . I have been able to set most options directly through command line options. this is the one case  where that is not helping, and it might just be that I don't know how. Let me see if I am able to get this working.

So does Libmesh create the PETSc solver? If so, it should be calling KSPSetDM() or SNESSetDM() or TSSetDM(). This

is all I want to know. It seems like this is not the case since the PCFIELDSPLIT says it has no fields. Either that, or they have

a bug in the DMCreateFieldDecomposition() implementation.

Either way, it seems like the thing to do is run with -start_in_debugger, and break in PCFieldSplitSetDefaults() where that is called.

    Matt

Thanks, 

Subramanya 

Date: Sat, 20 Jul 2013 06:07:47 -0500

Subject: Re: [petsc-users] Trying to set up a field-split preconditioner

From: knepley at gmail.com

To: potaman at outlook.com

CC: libmesh-users at lists.sourceforge.net; petsc-users at mcs.anl.gov

On Fri, Jul 19, 2013 at 11:50 PM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi Matt, 

I see that there is an implementation of the interface to DMCreateFieldDecomposition.html .  

So does this sound right?

Yes, that should automatically provide the field decomposition. Since this is not happening, something is

wrong in the code. Is the DM set in your solver?

1. I get index sets ,and variable names from the dm create field decomposition 

2. Once I have these , I create fieldsplits and name them using this.. 

3. And I guess I should be ready to go.. 

One question that remains is that the fieldsplit is created on a full matrix. However, an algorithm such as VIRS operates only on a subset of this full DM. Will the fieldsplit and preconditioner created on the full DM carry over to the subsidiary DMs?

VI is still new, and I have not tested in this case, but it is supposed to work.

  Thanks,

      Matt

Thanks for all the help!

Subramanya 

Date: Fri, 19 Jul 2013 23:09:10 -0500

Subject: Re: [petsc-users] Trying to set up a field-split preconditioner

From: knepley at gmail.com

To: potaman at outlook.com

CC: petsc-users at mcs.anl.gov

On Fri, Jul 19, 2013 at 9:59 PM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi Matt, 

The DM being created is here (this is from Libmesh code 

(petscdmlibmesh.C ) 

   01047 { 

   01048 PetscErrorCode ierr;

   01049 PetscFunctionBegin;

   01050 ierr = DMCreate(comm, dm); CHKERRQ(ierr);

   01051 ierr = DMSetType(*dm, DMLIBMESH); CHKERRQ(ierr);

   01052 ierr = DMLibMeshSetSystem(*dm, sys); CHKERRQ(ierr);

   01053 PetscFunctionReturn(0);

   01054 }

This file has methods to access the  variables assigned to the DM (this seems to be stored in a struct.) 

So , I guess one should be able to add a bit of code to create sections as you mentioned somewhere around here. 

Okay, they have their own DM. It must implement one of the interfaces for field specification. They could provide

  http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMCreateFieldDecomposition.html

or at a lower level

  http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMCreateSubDM.html

which in turn can be constructed by specifying a default PetscSection

  http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMSetDefaultSection.html

    Matt

Thanks, 

Subramanya 

Date: Fri, 19 Jul 2013 21:33:11 -0500

Subject: Re: [petsc-users] Trying to set up a field-split preconditioner

From: knepley at gmail.com

To: potaman at outlook.com

CC: petsc-users at mcs.anl.gov

On Fri, Jul 19, 2013 at 9:17 PM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi Matt, 

I am using Libmesh so the DM stuff is actually in the background, and unfortunately the matrix doesn't have a saddle point, 

I thought that 

  -ch_solve_fieldsplit_block_size 2

  -ch_solve_fieldsplit_0_fields 0

  -ch_solve_fieldsplit_1_fields 1

The block_size argument presumes you are using a DA. Are you?

The other two options just say select the first DM field as field 0 in this PC, and the same with the second field. The

DM must inform the PC about the initial field decomposition.

would inform the solver of the structure. If this doesn't work owing to the fact that the problem is only being solved on a section of the mesh (because of the reduced space method), I guess I will have to use the PetscSection. Does that sound right?

First, I think the right people to do this are the Libmesh people (we will of course help them). Second, you have not said

whether you are using a structured or unstructured mesh. What DM class does the solver actually see?

  Thanks,

      Matt

Thanks, 

Subramanya 

Subject: Re: [petsc-users] Trying to set up a field-split preconditioner

From: knepley at gmail.com

To: potaman at outlook.com

CC: petsc-users at mcs.anl.gov; libmesh-users at lists.sourceforge.net

On Fri, Jul 19, 2013 at 7:56 PM, subramanya sadasiva <potaman at outlook.com> wrote:

Hi, 

I am trying to set up a fieldsplit preconditioner for my Cahn Hilliard solver and I get the following error, 

You have to tell the PCFIELDSPLIT about the dofs in each field. So

1) You are probably not using a DA, since it would tell it automatically

2) If you have a saddle point, you can use -pc_fieldsplit_detect_saddle_point

3) If none of those apply, you can set a PetscSection describing your layout to the DM for the solver.

     Since this is new, I suspect you will need help, so mail back.

  Thanks,

     Matt

[0]PETSC ERROR: --------------------- Error Message ------------------------------------

[0]PETSC ERROR: Petsc has generated inconsistent data!

[0]PETSC ERROR: Unhandled case, must have at least two fields, not 0!

[0]PETSC ERROR: ------------------------------------------------------------------------

These are the options that I am using, 

-ch_solve is just a prefix. 

-ch_solve_pc_type fieldsplit

-ch_solve_pc_fieldsplit_type schur

-ch_solve_fieldsplit_block_size 2

-ch_solve_fieldsplit_0_field 1

-ch_solve_fieldsplit_1_field 0

-ch_solve_fieldsplit_0_ksp_type cg 

-ch_solve_fieldsplit_0_pc_type hypre

-ch_solve_fieldsplit_0_pc_type_hypre boomeramg 

-ch_solve_fieldsplit_1_ksp_type cg

-ch_solve_fieldsplit_1_pc_type hypre 

-ch_solve_fieldsplit_1_pc_type_hypre boomeramg

Any ideas?

Thanks,

Subramanya 

--