[petsc-users] Wrong Global Vector size when default section is built after dmdistribute?

[Blaise A Bourdin]

Of course, you are right.
In retrospect, trying to create the natural to global SF during distribution does not seem to be the smartest idea…
Would you have time to take a pass at distributing the constrains?
On our side, we will work on patch that separates the creation of the Naturaltoglobal SF from distribution, and works when the default section was created on the sequential or distributed DM.
It would just be noce to have both cases working

Regards,
Blaise



On Feb 23, 2019, at 2:39 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:

On Sat, Feb 23, 2019 at 10:48 AM Blaise A Bourdin via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Hi,

My student Alex and I are trying to build an example combining natural to global ordering and constraints.
Constraints information do not seem to be distributed,

Well dang. That is true. I never considered doing it the other way since all my specifications come from compact things
like an FEM discretization and BC labels, which get distributed. But yes, this is a big hole. I think it could be fixed without
too much work since the constraints are just another section+IS to distribute, and the IS does not even have to be renumbered.
I think we can just call https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/DMPlexDistributeFieldIS.html on it.

so the most rational thing to do seems to be creating the default section after distribution, then figuring out a way to reconstruct the natural to global stuff.

That is what I do, but we could do it the other way.

In the example attached, we do this, then create a global vector, but for some reason that I do not understand, the global vector size is 0 (should be 18) when run on more than 1 CPU. If we make 2 calls to DMGetGlobalVector, we get a vector of the proper size. What are we doing wrong?

At the beginning, you turn on

  DMSetUseNatural(dm, PETSC_TRUE);

so during Distribute() it tries to make the mapping. This process creates a global vector, which has size 0, because
the Section is empty. The reason it works when you call it again is that it creates a new vector this time, now with
the new Section. So, to do it this way, don't set this flag at the beginning. You have to setup the GlobalToNatural() afterwards.

Also, this shows that I should probably invalidate the global/local vector cache after a new Section is set.

  Thanks,

    Matt

Regards,
Blaise


--
Department of Mathematics and Center for Computation & Technology
Louisiana State University, Baton Rouge, LA 70803, USA
Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin









--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>

--
Department of Mathematics and Center for Computation & Technology
Louisiana State University, Baton Rouge, LA 70803, USA
Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190225/5ddaa10f/attachment.html>