[petsc-users] DMSTAG Gathering Vector on single process

[Matthew Knepley]

On Wed, Dec 6, 2023 at 5:54 PM Colton Bryant <
coltonbryant2021 at u.northwestern.edu> wrote:

> Hello,
>
> I am working on a code in which a DMSTAG object is used to solve a fluid
> flow problem and I need to gather this flow data on a single process to
> interact with an existing (serial) library at each timestep of my
> simulation. After looking around the solution I've tried is:
>
> -use DMStagVecSplitToDMDA to extract vectors of each component of the flow
> -use DMDACreateNaturalVector and DMDAGlobalToNatural to get the components
> naturally ordered
> -use VecScatterCreateToZero to set up and then do the scatter to gather on
> the single process
>
> Unless I'm misunderstanding something this method results in a lot of
> memory allocation/freeing happening at each step of the evolution and I was
> wondering if there is a way to directly perform such a scatter from the
> DMSTAG object without splitting as I'm doing here.
>

1) You can see here:

https://petsc.org/main/src/dm/impls/stag/stagda.c.html#DMStagVecSplitToDMDA

that this function is small. You can do the DMDA creation manually, and
then just call DMStagMigrateVecDMDA() each time, which will not create
anything.

2) You can create the natural vector upfront, and just scatter each time.

3) You can create the serial vector upfront, and just scatter each time.

This is some data movement. You can compress the g2n and 2zero scatters
using

  https://petsc.org/main/manualpages/PetscSF/PetscSFCompose/

as an optimization.

  Thanks,

     Matt


> Any advice would be much appreciated!
>
> Best,
> Colton Bryant
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20231206/16147b7c/attachment.html>