[petsc-users] Ghost points in DMSwarm

[Matthew Knepley]

On Sun, Feb 25, 2024 at 1:00 PM MIGUEL MOLINOS PEREZ <mmolinos at us.es> wrote:

> I see, so I'm the one in charge of the synchronization between the
> particles and the replicas associated, right?
>

Yes. We could conceivably put this in during our rewrite of DMSwarm, but it
would change the semantics. Right now, there is no particle identity.

  Thanks,

    Matt


> Best,
> Miguel
>
> On Feb 25, 2024, at 6:53 PM, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Sun, Feb 25, 2024 at 7:21 AM MIGUEL MOLINOS PEREZ <mmolinos at us.es>
> wrote:
>
>> Dear Matthew,
>>
>> Thank you for answer me out of office hours and for the suggestion. If I
>> interpret it correctly, your point is using the background mesh halo to
>> identify and create ghosts particles in the overlapping region using
>> DMSwarmMigrate().
>>
>> Then two questions hits me:
>>
>>    - If I change a variable in the “real” particle (say the position),
>>    it should be updated in the replicas on the overlapped regions, right? This
>>    sort of communicator is included in DMSwarmMigrate()?
>>
>> No, the particles that are migrated are copies. There is no current
> analogue of DMGlobalToLocal(). It would need to be constructed. It is
> possible, but would take writing some code. All applications we have
> supported to date do not use replicated particles.
>
>>
>>    - The function DMSwarmCreateLocalVectorFromField() does not include
>>    extra space for ghost particles. Therefore, I should resize the local size
>>    for each rank to accommodate the replicas manually. Right?
>>
>> It uses all particles on the process. If some of those are replicas, they
> will be included.
>
>   Thanks,
>
>      Matt
>
>
>> Anyway, I think this this will take me more time I expected originally.
>> Meanwhile,  I will go ahead with my “humble” parallelization scheme which
>> consist in using AllReduce to create a serial vectors with the variables I
>> need to evaluate rank-wise the residual equations of my problem  (please,
>> don’t judge me, publish or perish...)
>>
>> Many thanks,
>> Miguel
>>
>>
>> On Feb 24, 2024, at 4:24 PM, Matthew Knepley <knepley at gmail.com> wrote:
>>
>> On Fri, Feb 23, 2024 at 5:14 PM MIGUEL MOLINOS PEREZ <mmolinos at us.es>
>> wrote:
>>
>>> Dear all, I am struggling on how to include ghost points in DMSwarm
>>> local vectors. According to PETSc documentation it seems “straightforward”
>>> for a DMDA mesh. However, I am not so sure on how to do it for a DMSwarm.
>>> In fact, if I add the result
>>> ZjQcmQRYFpfptBannerStart
>>> This Message Is From an External Sender
>>> This message came from outside your organization.
>>>
>>> ZjQcmQRYFpfptBannerEnd
>>> Dear all,
>>>
>>> I am struggling on how to include ghost points in DMSwarm local vectors.
>>> According to PETSc documentation it seems “straightforward” for a DMDA
>>> mesh. However, I am not so sure on how to do it for a DMSwarm. In fact,
>>> if I add the result of DMSwarmGetLocalSize for each rank, the result is
>>> the exact number of particles in the system. Which means that the halo
>>> is zero.
>>>
>>> Since there is a function called DMSwarmCreateLocalVectorFromField,I
>>> was wandering if there is a function already implemented in petsc (and
>>> I’m missing it) to include ghost points in DMSwarm and therefore don’t
>>> have to reinvent the wheel. If so, is there any example out there I can
>>> follow?
>>>
>>
>> DMSwarm is currently different from the other DMs. There is no idea of
>> identity for particles, so there is no idea of a shared particle. You can
>> create this by adding an ID to your particle fields, but it would be a
>> manual process. DMSwarmMigrate() can duplicate particles if you have an
>> overlapping mesh, which would mimic shared particles.
>>
>>   THanks,
>>
>>      Matt
>>
>>
>>> Thanks in advance.
>>>
>>> Regards,
>>> Miguel
>>>
>>> petsc.org
>>> <https://urldefense.us/v3/__https://petsc.org/release/manual/vec/__;!!G_uCfscf7eWS!f6U_z6q8Rk6XrOxvbmE8KyNErlmqYwpGzkxCQ56xX0agWaCG0tLVLh1Cml6fTtqvve0aL3HGiAhZn-hDgIoH5w$>
>>>
>>> <https://urldefense.us/v3/__https://petsc.org/release/manual/vec/__;!!G_uCfscf7eWS!f6U_z6q8Rk6XrOxvbmE8KyNErlmqYwpGzkxCQ56xX0agWaCG0tLVLh1Cml6fTtqvve0aL3HGiAhZn-hDgIoH5w$>
>>> <https://urldefense.us/v3/__https://petsc.org/release/manual/vec/__;!!G_uCfscf7eWS!f6U_z6q8Rk6XrOxvbmE8KyNErlmqYwpGzkxCQ56xX0agWaCG0tLVLh1Cml6fTtqvve0aL3HGiAhZn-hDgIoH5w$>
>>>
>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKXzs_I05$ 
>> <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKeijRazj$ >
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKXzs_I05$ 
> <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKeijRazj$ >
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKXzs_I05$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eDmkxFmYvuRK7K4mCXY1o6YNM-qQM7ujn4OiPKemscUGOZ5aL9E_0BY0NgwxYeuNOJ8AZTj4c29sKeijRazj$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240225/fcf2e2a4/attachment-0001.html>