<div dir="ltr"><div dir="ltr">On Fri, Mar 13, 2020 at 9:45 AM Berend van Wachem <<a href="mailto:berend.vanwachem@ovgu.de">berend.vanwachem@ovgu.de</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear Matt,<br>
<br>
Thanks for your response. My understanding of the DM and DMClone is the <br>
same - and I have tested this with a DMPLEX DM without problems.<br>
<br>
However, for some reason, I cannot change/set the section of a P4EST dm. <br>
In the attached example code, I get an error in line 140, where I try to <br>
create a new section from the cloned P4EST DM. Is it not possible to <br>
create/set a section on a P4EST DM? Or maybe I am doing something else <br>
wrong? Do you suggest a workaround?<br></blockquote><div><br></div><div>Hi Berend,</div><div><br></div><div>Sorry I am behind. The problem on line 140 is that you call a DMPlex function (DMPlexCreateSection)</div><div>with a DMForest object. That is illegal. You can, however, call that function using the Plex you get from</div><div>a Forest using DMConvert(DMForestClone, DMPLEX, &plexClone). I will get your code running as soon</div><div>as I can, but after you create the Section, attaching it should be fine.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Many thanks, Berend.<br>
<br>
<br>
On 2020-03-13 00:19, Matthew Knepley wrote:<br>
> On Thu, Mar 12, 2020 at 7:40 AM Berend van Wachem <br>
> <<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a> <mailto:<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a>>> wrote:<br>
> <br>
> Dear All,<br>
> <br>
> I have started to use DMPLEX with P4EST for a computational fluid<br>
> dynamics application. I am solving a coupled system of 4 discretised<br>
> equations (for 3 velocity components and one pressure) on a mesh.<br>
> However, next to these 4 variables, I also have a few single field<br>
> variables (such as density and viscosity) defined over the mesh,<br>
> which I<br>
> don't solve for (they should not be part of the matrix with unknowns).<br>
> Most of these variables are at the cell centers, but in a few cases, it<br>
> want to define them at cell faces.<br>
> <br>
> With just DMPLEX, I solve this by:<br>
> <br>
> DMPlexCreateMesh, so I get an initial DM<br>
> DMPlexCreateSection, indicating the need for 4 variables<br>
> DMSetLocalSection<br>
> DMCreateGlobalVector (and Matrix), so I get an Unknown vector, a RHS<br>
> vector, and a matrix for the 4 variables.<br>
> <br>
> To get a vector for a single variable at the cell center or the cell<br>
> face, I clone the original DM, I define a new Section on it, and then<br>
> create the vector from that which I need (e.g. for density,<br>
> viscosity or<br>
> a velocity at the cell face).<br>
> <br>
> Then I loop over the mesh, and with MatSetValuesLocal, I set the<br>
> coefficients. After that, I solve the system for multiple timesteps<br>
> (sequential solves) and get the solution vector with the 4 variables<br>
> after each solve.<br>
> <br>
> So-far, this works fine with DMPLEX. However, now I want to use P4EST,<br>
> and I have difficulty defining a variable vector other than the<br>
> original 4.<br>
> <br>
> I have changed the code structure:<br>
> <br>
> DMPlexCreateMesh, so I get an initial DM<br>
> DMPlexCreateSection, indicating the need for 4 variables<br>
> DMSetLocalSection<br>
> DMForestSetBaseDM(DM, DMForest) to create a DMForest<br>
> DMCreateGlobalVector (and Matrix), so I get a Unknown vector, a RHS<br>
> vector, and a matrix for the 4 variables<br>
> <br>
> then I perform multiple time-steps,<br>
> DMForestTemplate(DMForest -> DMForestPost)<br>
> Adapt DMForestPost<br>
> DMCreateGlovalVector(DMForestPost, RefinedUnknownVector)<br>
> DMForestTransferVec(UnknownVector , RefinedUnknownVector)<br>
> DMForestPost -> DMForest<br>
> and then DMConvert(DMForest,DMPLEX,DM)<br>
> and I can solve the system as usual. That also seems to work.<br>
> <br>
> But my conceptual question: how can I convert the other variable<br>
> vectors<br>
> (obtained with a different section on the same DM) such as density and<br>
> viscosity and faceVelocity within this framework?<br>
> <br>
> <br>
> Here is my current thinking about DMs. A DM is a function space <br>
> overlaying a topology. Much to my dismay, we<br>
> do not have a topology object, so it hides inside DM. DMClone() creates <br>
> a shallow copy of the topology. We use<br>
> this to have any number of data layouts through PetscSection, laying <br>
> over the same underlying topology.<br>
> <br>
> So for each layout you have, make a separate clone. Then things like <br>
> TransferVec() will respond to the layout in<br>
> that clone. Certainly it works this way in Plex. I admit to not having <br>
> tried this for TransferVec(), but let me know if<br>
> you have any problems.<br>
> <br>
> BTW, I usually use a dm for the solution, which I give to the solver, <br>
> say SNESSetDM(snes, dm), and then clone<br>
> it as dmAux which has the layout for all the auxiliary fields that are <br>
> not involved in the solve. The Plex examples<br>
> all use this form.<br>
> <br>
> Thanks,<br>
> <br>
> Matt<br>
> <br>
> The DMForest has the same Section as the original DM and will thus have<br>
> the space for exactly 4 variables per cell. I tried pushing another<br>
> section on the DMForest and DMForestPost, but that does not seem to<br>
> work. Please find attached a working example with code to do this,<br>
> but I<br>
> get the error:<br>
> <br>
> PETSC ERROR: PetscSectionGetChart() line 513 in<br>
> /usr/local/petsc-3.12.4/src/vec/is/section/interface/section.c Wrong<br>
> type of object: Parameter # 1<br>
> <br>
> So, I is there a way to DMForestTransferVec my other vectors from one<br>
> DMForest to DMForestPost. How can I do this?<br>
> <br>
> Many thanks for your help!<br>
> <br>
> Best wishes, Berend.<br>
> <br>
> <br>
> <br>
> -- <br>
> What most experimenters take for granted before they begin their <br>
> experiments is infinitely more interesting than any results to which <br>
> their experiments lead.<br>
> -- Norbert Wiener<br>
> <br>
> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>