[petsc-users] Interpreting Redistribution SF

[Nicholas Arnold-Medabalimi]

Hi Matt

Yep, that makes sense and is consistent.

My question is a little more specific. So let's say I take an initial mesh
and distribute it and get the distribution SF with an overlap of one.
Consider a cell that is a root on process 0 and a leaf on process 1 after
the distribution.

Will the distribution pointSF have an entry for the cell that is a leaf in
the ghost cell sense?

I guess, in short does the distribution SF only have entries for the
movement of points that are roots in the ghost SF?

Sorry if this is a little unclear.

Maybe my usage will be a bit clearer. I am generating a distributionSF
(type 2 in your desc) then using that to generate a dof distribution(type
3) using the section information. I then pass the information from the
initial distribution to new distribution with PetscSFBcast with
MPI_REPLACE. That scatters the vector to the new distribution. I then do
"stuff" and now want to redistribute back. So I pass the same dof
distributionSF but call PetscSFReduce with MPI_REPLACE. My concern is I am
only setting the root cell values on each partition. So if the ghost cells
are part of the distribution SF there will be multiple cells reducing to
the original distribution cell?


Thanks
Nicholas


On Thu, Jan 19, 2023 at 8:28 PM Matthew Knepley <knepley at gmail.com> wrote:

> On Thu, Jan 19, 2023 at 11:58 AM Nicholas Arnold-Medabalimi <
> narnoldm at umich.edu> wrote:
>
>> Hi Petsc Users
>>
>> I'm working with a distribution start forest generated by
>> DMPlexDistribute and PetscSFBcast and Reduce to move data between the
>> initial distribution and the distribution generated by DMPlex Distribute.
>>
>> I'm trying to debug some values that aren't being copied properly and
>> wanted to verify I understand how a redistribution SF works compared with a
>> SF that describes overlapped points.
>>
>>   [0] 0 <- (0,7)     point 0 on the distributed plex is point 7 on
>> process 0 on the initial distribution
>>   [0] 1 <- (0,8)     point 1 on the distributed plex is point 8 on
>> process 0 on the initial distribution
>>   [0] 2 <- (0,9)
>>   [0] 3 <- (0,10)
>>   [0] 4 <- (0,11)
>>
>>   [1] 0 <- (1,0)     point 0 on the distributed plex is point 0 on
>> process 1 on the initial distribution
>>   [1] 1 <- (1,1)
>>   [1] 2 <- (1,2)
>>   [1] 3 <- (0,0)     point 3 on the distributed plex is point 0 on
>> process 0 on the initial distribution
>>   [1] 4 <- (0,1)
>>   [1] 5 <- (0,2)
>>
>>  my confusion I think is how does the distributionSF inform of what cells
>> will be leafs on the distribution?
>>
>
> I should eventually write something to clarify this. I am using SF in (at
> least) two different ways.
>
> First, there is a familiar SF that we use for dealing with "ghost" points.
> These are replicated points where one process
> is said to "own" the point and another process is said to hold a "ghost".
> The ghost points are leaves in the SF which
> point back to the root point owned by another process. We call this the
> pointSF for a DM.
>
> Second, we have a migration SF. Here the root points give the original
> point distribution. The leaf points give the new
> point distribution. Thus a PetscSFBcast() pushes points from the original
> to new distribution, which is what we mean
> by a migration.
>
> Third, instead of point values, we might want to communicate fields over
> those points. For this we make new SFes,
> where the numbering does not refer to points, but rather to dofs.
>
> Does this make sense?
>
>   Thanks,
>
>     Matt
>
>
>> Sincerely
>> Nicholas
>>
>> --
>> Nicholas Arnold-Medabalimi
>>
>> Ph.D. Candidate
>> Computational Aeroscience Lab
>> University of Michigan
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>


-- 
Nicholas Arnold-Medabalimi

Ph.D. Candidate
Computational Aeroscience Lab
University of Michigan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230119/cbdb69b8/attachment.html>