[petsc-users] extract arbitrary subset of a DMDA

[Matthew Knepley]

On Fri, Oct 10, 2025 at 11:15 AM Matteo Semplice <
matteo.semplice at uninsubria.it> wrote:

> On 10/10/2025 16:39, Matthew Knepley wrote:
>
> On Fri, Oct 10, 2025 at 9:48 AM Matteo Semplice via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Dear all,
>>
>>      I am wondering if there is a way to extract a subset of a DMDA and
>> use it as a mesh. The use case is to program a finite-difference method
>> in which the domain is defined by a levelset function: if I could
>> completely ignore the parts of the background DMDA that are "far away"
>> from the object, I guess I would avoid some cores having almost no
>> workload. I figure that I could setup a DMDA, load/compute the levelset
>> on the entire box, then mark the nodes to be retained, extract the
>> submesh and repartition it. I would also need a mean to transfer some
>> Vec data from the DMDA to the new mesh.
>>
>> I guess that the extracted mesh would then become a DMPlex and it would
>> not retain any DMDA flavour (like notions of which are the grid nodes
>> sitting on top/bottom, left/right of a given node), right?
>>
>
> If you are planning on extracting a Plex anyway, I think it would be
> easier to just
> start with a Cartesian Plex, instead of a DA, and use
> DMPlexCreateSubmesh().
>
> Hmmm... doable, but I have a couple of questions.
>
> By Cartesian Plex you mean a Plex created by DMPlexCreateBoxMesh with
> simplex=false, right?
>
Yes, exactly

> And, could you point me to the routines that can perform data tranfer from
> Vecs associated to the DM to the ones asscoiated to the subDM? Is
> DMPlexGetSubpointIS the way to go?
>
Yes. You get the IS and then you can use

  https://urldefense.us/v3/__https://petsc.org/main/manualpages/Vec/VecISCopy/__;!!G_uCfscf7eWS!ZuU3KyjiAxEurYfyBPAqLwse3wiBnKnWg7CgVwR0lJDOM2Rp3loC13vFnsgCTd7FcZMhSe0CAY8ingyIUWji$ 

> Next, I will load the levelsets from the output of another code that is
> DA-based and that I'd really like to reuse some code in the setup phase
> which relies on the DA indexing. So maybe I'd rather, create the DMDA and
> the associated Vecs, do the setup phase, then DMConvert the DMDA to a
> "large" DMPlex that covers the entire box, transfer the DA Vecs to the
> "large" Plex vectors and then extract the submesh. Would this be feasible?
> If so, can you point me to the routines to transfer the vecs from the dmda
> to the large plex?
>
Yes, this sounds doable, and once we write it, we should just put it in the
library.

In serial this is completely trivial. The DA has a known ordering and the
Plex has a known ordering. We can use an AO
or just a VecScatter to permute the Vec. In parallel, the DA is partitioned
geometrically, whereas the Plex, by default, is partitioned using a graph
partitioner, like ParMetis. To me, if this project is important, it seems
worth it to build a simple partitioner for the Plex that mimics the DA.
Then we are back to the trivial remapping. Finally, we probably want the
Plex dual to the DA due to the way that DAs partition things.

  Thanks,

     Matt

> Thanks
>
>     Matteo
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZuU3KyjiAxEurYfyBPAqLwse3wiBnKnWg7CgVwR0lJDOM2Rp3loC13vFnsgCTd7FcZMhSe0CAY8inhPGfPNx$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZuU3KyjiAxEurYfyBPAqLwse3wiBnKnWg7CgVwR0lJDOM2Rp3loC13vFnsgCTd7FcZMhSe0CAY8inlmYDKBF$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20251011/bbeead5b/attachment.html>