<div dir="ltr"><div dir="ltr"><div dir="ltr">Sorry, I forgot three more lines describing the 3rd level of split:<div><br></div><div><div>-pc_type fieldsplit</div><div>-pc_fieldsplit_0_fields 0,1,2</div><div>-pc_fieldsplit_1_fields 3,4,5</div><div>-pc_fieldsplit_2_fields 6,7,8</div><div>-fieldsplit_0_pc_type fieldsplit</div><div>-fieldsplit_1_pc_type fieldsplit</div><div>-fieldsplit_2_pc_type fieldsplit</div><div>-fieldsplit_0_pc_fieldsplit_0_fields 0,1</div><div>-fieldsplit_0_pc_fieldsplit_1_fields 2</div><div>-fieldsplit_1_pc_fieldsplit_0_fields 3,4</div><div>-fieldsplit_1_pc_fieldsplit_1_fields 5</div><div>-fieldsplit_2_pc_fieldsplit_0_fields 6,7</div><div>-fieldsplit_2_pc_fieldsplit_1_fields 8</div></div><div><br></div><div><div>-fieldsplit_0_fieldsplit_0_pc_type fieldsplit</div><div>-fieldsplit_1_fieldsplit_0_pc_type fieldsplit</div><div>-fieldsplit_2_fieldsplit_0_pc_type fieldsplit</div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jan 2, 2019 at 2:54 PM Justin Chang <<a href="mailto:jychang48@gmail.com">jychang48@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Okay that's fine.<div><br></div><div>Now consider one more case:</div><div><br></div><div>Suppose I create a PetscSection with 9 fields:</div><div><br></div><div><div>section = PETSc.Section().create()</div><div>section.setNumFields(9)</div><div>section.setFieldName(0,'u1')</div><div>section.setFieldName(1,'c1')</div><div>section.setFieldName(2,'t1')</div><div>section.setFieldName(3,'u2')</div><div>section.setFieldName(4,'c2')</div><div>section.setFieldName(5,'t2')</div><div>section.setFieldName(6,'u3')</div><div>section.setFieldName(7,'c3')</div><div>section.setFieldName(8,'t3')</div><div>section.setFieldComponents(0,1)</div><div>section.setFieldComponents(1,1)</div><div>section.setFieldComponents(2,1)</div><div>section.setFieldComponents(3,1)</div></div><div>section.setFieldComponents(4,1)</div><div>section.setFieldComponents(5,1)</div><div>section.setFieldComponents(6,1)</div><div>section.setFieldComponents(7,1)</div>section.setFieldComponents(8,1)</div><div dir="ltr"><br></div><div dir="ltr">....</div><div dir="ltr"><br></div><div>where u is potential, c is concentration, and t is temperature. The numbers refer to different domains of a battery (anode, electrolyte, cathode).</div><div dir="ltr"><br></div><div>I want three levels of splits:</div><div>1) Split by domain</div><div>2) Split potential and concentration from temperature.</div><div>3) Split potential and concentration for schur complementing purpose.</div><div><br></div><div>Do the following PETSc command line options describe the split I mentioned above:</div><div><br></div><div><div>-pc_type fieldsplit</div><div>-pc_fieldsplit_0_fields 0,1,2</div><div>-pc_fieldsplit_1_fields 3,4,5</div><div>-pc_fieldsplit_2_fields 6,7,8</div><div>-fieldsplit_0_pc_type fieldsplit</div><div>-fieldsplit_1_pc_type fieldsplit</div><div>-fieldsplit_2_pc_type fieldsplit</div><div>-fieldsplit_0_pc_fieldsplit_0_fields 0,1</div><div>-fieldsplit_0_pc_fieldsplit_1_fields 2</div><div>-fieldsplit_1_pc_fieldsplit_0_fields 3,4</div><div>-fieldsplit_1_pc_fieldsplit_1_fields 5</div><div>-fieldsplit_2_pc_fieldsplit_0_fields 6,7</div><div>-fieldsplit_2_pc_fieldsplit_1_fields 8</div></div><div><br></div><div>I don't have an actual code to test these out with yet as I've never done anything beyond two levels of split. So I am wondering if the above will break anything as far as you can tell.</div><div><br></div><div>Thanks,</div><div>Justin</div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jan 1, 2019 at 7:10 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Tue, Jan 1, 2019 at 5:06 AM Justin Chang <<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Okay so I managed to successfully do this by translating the IS's into a PetscSection, assigning it to an empty DMShell, and then assigning that to KSP. <div><br></div><div>However, I seem to be unable to rename the outer fields that aggregate two or more inner fields.</div></div></div></div></div></blockquote><div><br></div><div>Right now you cannot do that because of the way that I check for the option:</div><div><br></div><div><a href="https://bitbucket.org/petsc/petsc/src/0a5f382fff62a328ec919e3e9fe959e6cbbcf413/src/ksp/pc/impls/fieldsplit/fieldsplit.c#lines-369" target="_blank">https://bitbucket.org/petsc/petsc/src/0a5f382fff62a328ec919e3e9fe959e6cbbcf413/src/ksp/pc/impls/fieldsplit/fieldsplit.c#lines-369</a><br></div><div><br></div><div>I guess we could replace this by some search code that looked for any option of that form and then read</div><div>out the splitname using scanf. Right now, I do not see a pattern search function for the options database.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div> Consider this snippet of a four-field dual porosity/permeability FEniCS/petsc4py code:</div><div><br></div><div>...</div><div><br></div><div><div>## Extract FEniCS dof layout, global indices ##</div><div>dof_total = np.array(W.dofmap().dofs())</div><div>dof_v1 = np.array(W.sub(0).dofmap().dofs())</div><div>dof_p1 = np.array(W.sub(1).dofmap().dofs())</div><div>dof_v2 = np.array(W.sub(2).dofmap().dofs())</div><div>dof_p2 = np.array(W.sub(3).dofmap().dofs())</div><div>offset = np.min(dof_total)</div><div><br></div><div>## Create PetscSection ##</div><div>section = PETSc.Section().create()</div><div>section.setNumFields(4)</div><div>section.setFieldName(0,'v1')</div><div>section.setFieldName(1,'p1')</div><div>section.setFieldName(2,'v2')</div><div>section.setFieldName(3,'p2')</div><div>section.setFieldComponents(0,1)</div><div>section.setFieldComponents(1,1)</div><div>section.setFieldComponents(2,1)</div><div>section.setFieldComponents(3,1)</div><div>section.setChart(0,len(dof_total))</div><div>for i in np.nditer(dof_v1):</div><div> section.setDof(i-offset,1)</div><div> section.setFieldDof(i-offset,0,1)</div><div>for i in np.nditer(dof_p1):</div><div> section.setDof(i-offset,1)</div><div> section.setFieldDof(i-offset,1,1)</div><div>for i in np.nditer(dof_v2):</div><div> section.setDof(i-offset,1)</div><div> section.setFieldDof(i-offset,2,1)</div><div>for i in np.nditer(dof_p2):</div><div> section.setDof(i-offset,1)</div><div> section.setFieldDof(i-offset,3,1)</div><div>section.setUp()</div><div><br></div><div>## Create DM and assign PetscSection ##</div><div>dm = PETSc.DMShell().create()</div><div>dm.setDefaultSection(section)</div><div>dm.setUp()</div><div><br></div><div>## Create KSP and assign DM ##</div><div>ksp = PETSc.KSP().create()</div><div>ksp.setDM(dm)</div><div>ksp.setDMActive(False)</div></div><div><br></div><div><div>### PETSc Command-line options ##</div><div>PETScOptions.set('ksp_monitor_true_residual')</div><div>PETScOptions.set('ksp_view')</div><div>PETScOptions.set('ksp_type','gmres')</div><div>PETScOptions.set('pc_type','fieldsplit')</div><div>PETScOptions.set('pc_fieldsplit_0_fields','0,1')</div><div>PETScOptions.set('pc_fieldsplit_1_fields','2,3')</div><div>PETScOptions.set('pc_fieldsplit_type','additive')</div><div>PETScOptions.set('fieldsplit_0_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_0_pc_type','fieldsplit')</div><div>PETScOptions.set('fieldsplit_0_pc_fieldsplit_type','schur')</div><div>PETScOptions.set('fieldsplit_0_pc_fieldsplit_schur_fact_type','full')</div><div>PETScOptions.set('fieldsplit_0_pc_fieldsplit_schur_precondition','selfp')</div><div>PETScOptions.set('fieldsplit_0_fieldsplit_v1_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_0_fieldsplit_v1_pc_type','bjacobi')</div><div>PETScOptions.set('fieldsplit_0_fieldsplit_p1_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_0_fieldsplit_p1_pc_type','hypre')</div><div>PETScOptions.set('fieldsplit_1_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_1_pc_type','fieldsplit')</div><div>PETScOptions.set('fieldsplit_1_pc_fieldsplit_type','schur')</div><div>PETScOptions.set('fieldsplit_1_pc_fieldsplit_schur_fact_type','full')</div><div>PETScOptions.set('fieldsplit_1_pc_fieldsplit_schur_precondition','selfp')</div><div>PETScOptions.set('fieldsplit_1_fieldsplit_v2_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_1_fieldsplit_v2_pc_type','bjacobi')</div><div>PETScOptions.set('fieldsplit_1_fieldsplit_p2_ksp_type','preonly')</div><div>PETScOptions.set('fieldsplit_1_fieldsplit_p2_pc_type','hypre')</div></div><div><br></div><div>...</div><div><br></div><div>Is it possible to rename the outer splits (aka make '-pc_fieldsplit_0_fields 0,1' be something like '-pc_fieldsplit_micro_fields 0,1')?</div><div><br></div><div>Thanks,</div><div>Justin</div><div><br></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 31, 2018 at 7:40 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Mon, Dec 31, 2018 at 2:40 AM Justin Chang via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi all,<div><br></div><div>I am solving a six field battery problem (concentration and potential for each of the two solid and one electrolyte domains) and I want to experiment with nested/recursice fieldsplitting. I have the IS's and am able to use these to define my PCFieldSplitsSetIS(). However, I can imagine this getting really messy from a programming standpoint, especially once I need to add temperature into the mix, so it is my hope that I can translate these index sets and fields into a DM (maybe DMShell?) so that I can just rely on command line options to play around with various combinations of field assignments and splits (e.g. -pc_fieldsplit_X_fields)</div><div><br></div><div>However, it doesn't seem clear to me how I would create a DM when you already know the IS's for each fields. If I understand functions like DMCreateFieldDecomposition() correctly, it seems that it returns to you the IS's and sub DM's associated with the original DM, whereas I want to do it the other way around. Perhaps the "reversal" of something like <a href="https://www.mcs.anl.gov/petsc/petsc-dev/src/dm/interface/dm.c.html#DMCreateFieldIS" target="_blank">DMCreateFieldIS()</a>, where you convert the IS into a PetscSection or is there an easier/better way?</div><div><br></div><div>Any thoughts/help appreciated!</div></div></div></blockquote><div><br></div><div>Paul has recently done this for LibMesh. I believe that constructing a PetscSection is enough to get you minimally started. That allows</div><div>DMCreateSubDM() to work by subsetting the Section, and that should allow the command line to work. CreateFieldDecomposition() should</div><div>be removed I think.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Justin</div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_136148431427166573gmail-m_7855016733326816947gmail-m_-82713115884670586gmail-m_-2710989791032934418gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_136148431427166573gmail-m_7855016733326816947gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>