[petsc-dev] DMplex: Natural Ordering and subDM

[Jed Brown]

Matthew Knepley <knepley at gmail.com> writes:

> On Mon, Nov 27, 2017 at 8:08 PM, Jed Brown <jed at jedbrown.org> wrote:
>
>> I don't know the answer to your question (Matt?), but do you really need
>> to reorder the entire mesh or would it be sufficient to label your
>> points with their original numbering?
>
>
> Maybe I am wrong, but I think it amounts to the same thing. If we are going
> to output things in parallel,
> we would need to communicate to he writing process, which this essentially
> does.

Writing a label doesn't require redistribution of the mesh.  It's possible to do parallel IO.

> I thought I responded, but what has to happen is that the SF for this
> reordering must be decimated to
> produce a subSF for the subDM. Its just not simple, so I have not had the
> free time to work it out.
>
>    Matt
>
>
>> Blaise A Bourdin <bourdin at lsu.edu> writes:
>>
>> > Hi,
>> > I am fighting with natural ordering in DMplex with multi-fields / multi
>> components DM.
>> > I need to read / write files in a distribution-independent ordering. The
>> natural way to do so is to define the natural ordering as that of the file,
>> then call DMPlexGlobalToNaturalBegin/End before VecView:
>> > The pseudo code would be
>> >
>> >   DMPlexCreateFromFile(PETSC_COMM_WORLD,ifilename,interpolate,&dmUAS);
>> >   // create default section associated with a complicated layout of
>> multiple fields
>> >   // (U,\alpha,\sigma)
>> >   DMSetUseNatural(dmUAS,PETSC_TRUE);
>> >   DMPlexDistribute(dmUAS,0,&migrationSF,&dmDist);
>> >
>> >   DMGetGlobalVector(dmDist,&UAS);
>> >   DMGetGlobalVector(dmDist,&UASNatural);
>> >   DMPlexGlobalToNaturalBegin(dmDist,UAS,UASNatural);
>> >   DMPlexGlobalToNaturalEnd(dmDist,UAS,UASNatural);
>> >   // Do my I/O stuff using UASNatural
>> >
>> > If I try to do the same thing one field at a time, things break:
>> > Say that I get the subDM for the first field (U):
>> >
>> >   DMCreateSubDM(dmDist,1,&fieldU,&ISU,&dmU);
>> >
>> > Then dmU->useNatural is PETSC_FALSE and  I cannot compute dmU->sfnatural
>> using DMPlexCreateGlobalToNaturalSF since I do not have the “sequential”
>> version of dmU.
>> >
>> > What is the proper way to handle this situation? Creating sequential
>> subDMs and calling DMDistribute on each of them is ugly and possibly costly.
>> >
>> > Regards,
>> >
>> > Blaise
>> >
>> > --
>> > Department of Mathematics and Center for Computation & Technology
>> > Louisiana State University, Baton Rouge, LA 70803, USA
>> > Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 http://www.math.lsu.edu/~
>> bourdin
>>
>
>
>
> -- 
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>