[petsc-users] extract arbitrary subset of a DMDA

[Semplice Matteo]

Inviato da Outlook per Android<https://urldefense.us/v3/__https://aka.ms/AAb9ysg__;!!G_uCfscf7eWS!fqtSeAyod1b2Xvs6xIj79ibX2IOgMwkxFyGY_V3CUj0CvphYw7SOJfqGyHEwfjbE1bLwzgUgz8wZRvDWCT0gByX2LMcN2kOlOg_8bQ$ >

________________________________
Da: Matthew Knepley <knepley at gmail.com>
Inviato: Sabato, Ottobre 11, 2025 11:19:23 PM
A: Semplice Matteo <matteo.semplice at uninsubria.it>
Cc: PETSc <petsc-users at mcs.anl.gov>
Oggetto: Re: [petsc-users] extract arbitrary subset of a DMDA

On Fri, Oct 10, 2025 at 11:15 AM Matteo Semplice <matteo.semplice at uninsubria.it<mailto:matteo.semplice at uninsubria.it>> wrote:

On 10/10/2025 16:39, Matthew Knepley wrote:

On Fri, Oct 10, 2025 at 9:48 AM Matteo Semplice via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Dear all,

     I am wondering if there is a way to extract a subset of a DMDA and
use it as a mesh. The use case is to program a finite-difference method
in which the domain is defined by a levelset function: if I could
completely ignore the parts of the background DMDA that are "far away"
from the object, I guess I would avoid some cores having almost no
workload. I figure that I could setup a DMDA, load/compute the levelset
on the entire box, then mark the nodes to be retained, extract the
submesh and repartition it. I would also need a mean to transfer some
Vec data from the DMDA to the new mesh.

I guess that the extracted mesh would then become a DMPlex and it would
not retain any DMDA flavour (like notions of which are the grid nodes
sitting on top/bottom, left/right of a given node), right?

If you are planning on extracting a Plex anyway, I think it would be easier to just
start with a Cartesian Plex, instead of a DA, and use DMPlexCreateSubmesh().

Hmmm... doable, but I have a couple of questions.

By Cartesian Plex you mean a Plex created by DMPlexCreateBoxMesh with simplex=false, right?

Yes, exactly

And, could you point me to the routines that can perform data tranfer from Vecs associated to the DM to the ones asscoiated to the subDM? Is DMPlexGetSubpointIS the way to go?

Yes. You get the IS and then you can use

  https://urldefense.us/v3/__https://petsc.org/main/manualpages/Vec/VecISCopy/__;!!G_uCfscf7eWS!fqtSeAyod1b2Xvs6xIj79ibX2IOgMwkxFyGY_V3CUj0CvphYw7SOJfqGyHEwfjbE1bLwzgUgz8wZRvDWCT0gByX2LMcN2kP-fgVdgg$ 

Next, I will load the levelsets from the output of another code that is DA-based and that I'd really like to reuse some code in the setup phase which relies on the DA indexing. So maybe I'd rather, create the DMDA and the associated Vecs, do the setup phase, then DMConvert the DMDA to a "large" DMPlex that covers the entire box, transfer the DA Vecs to the "large" Plex vectors and then extract the submesh. Would this be feasible? If so, can you point me to the routines to transfer the vecs from the dmda to the large plex?

Yes, this sounds doable, and once we write it, we should just put it in the library.

In serial this is completely trivial. The DA has a known ordering and the Plex has a known ordering. We can use an AO
or just a VecScatter to permute the Vec. In parallel, the DA is partitioned geometrically, whereas the Plex, by default, is partitioned using a graph partitioner, like ParMetis. To me, if this project is important, it seems worth it to build a simple partitioner for the Plex that mimics the DA. Then we are back to the trivial remapping. Finally, we probably want the Plex dual to the DA due to the way that DAs partition things.

____
Hi Matt,
   doing some research on parallel Plex I found this thread on the list from 2020

https://lists.mcs.anl.gov/pipermail/petsc-users/2020-August/041934.html 

I will explore this way to create a basic cartesian Plex from my distributed da. I think that there are enough info there, but I will come back if I run into troubles.

Or do you suggest that I should instead study the private methods used in DMPlexCreateBoxMesh?

Thanks
  Matteo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20251013/f527b885/attachment.html>