[petsc-users] DMSwarm Initialization on a DMPlex (PIC scheme)

[Matthew Knepley]

On Mon, Mar 30, 2026 at 2:21 PM Miguel Molinos <m.molinos at upm.es> wrote:

> Thank you Matt
>
> Indeed the right way was to use the rank property of the SF.
>

Great! Sometime you will have to show me some of the results.

  Thanks,

    Matt


> Thanks,
> Miguel
>
> PetscCall(
> DMLocatePoints(FE_Mesh, coordinates, DM_POINTLOCATION_NONE, &cellSF));
> PetscCall(PetscSFGetGraph(cellSF, &nroots, NULL, NULL, &LA_sfcell));
>
> PetscCall(VecGetArray(coordinates, &coordinates_ptr));
>
> // Note: mean_q_ptr should be filled based on n_atoms_local increment
> for (PetscInt site_i = 0; site_i < n_atoms; site_i++) {
> // Check if the point is found in the local process
> if (LA_sfcell[site_i].rank == rank_MPI) {
> idx_ptr[n_atoms_local] = site_i;
> for (int alpha = 0; alpha < dim; alpha++) {
> mean_q_ptr[n_atoms_local * dim + alpha] =
> Simulation_file.mean_q[site_i * dim + alpha];
> }
> n_atoms_local++;
> }
> }
>
>
> On Mar 30, 2026, at 2:50 AM, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Sun, Mar 29, 2026 at 6:25 PM Miguel Molinos <m.molinos at upm.es> wrote:
>
>> Dear all,
>>
>> I am experiencing an issue where particles created with DMSwarm are being
>> increased across MPI ranks when initializing a DMSwarm over a periodic
>> DMPlex. I follow this procedure on every rank:
>>
>>    1.
>>
>>    Create a sequential vector with the global coordinates of all
>>    particles (18,000 atoms).
>>    2.
>>
>>    Call DMLocatePoints(FE_Mesh, coordinates, DM_POINTLOCATION_NONE,
>>    &cellSF) to find which particles belong to the local DMPlex partition. Then
>>    call PetscSFGetGraph.
>>    3.
>>
>>    Iterate through the results and add any particle with a valid cell
>>    index (index != DMLOCATEPOINT_POINT_NOT_FOUND) to the local DMSwarm.
>>
>> The sum of n_atoms_local across all ranks gives 18,740 instead of the
>> expected 18,000. It seems that particles located exactly on periodic
>> boundaries or MPI interfaces are being identified as "found" by multiple
>> ranks simultaneously. Does DMLocatePoints return points found in ghost/halo
>> cells by default when the DMPlex is periodic or distributed with overlap?
>>
> I do not think this should happen at periodic boundaries, but definitely
> if you have cell overlap it will happen. It iterates over all local cells
> to check particle location. You could discard those points by checking if
> the cell is in the point SF in Step 3.
>
>   Thanks,
>
>       Matt
>
>> Thanks,
>>
>> Miguel
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bxNF7CZVEFp0DL1eyWSI-q2mEvdEpUJBuYVryE4HyTVqCjf90sjlGjoC8DR4_2B411ZwzCyYUIDIydqguJ43$ 
> <https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!D9dNQwwGXtA!X58muDkArsK-iH2YdyGstzlwOwzuxXNE308JQOmlP5pY7V8oHcgLbmf1Qiazmqv1IdaROicSzLQR_o8$ >
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bxNF7CZVEFp0DL1eyWSI-q2mEvdEpUJBuYVryE4HyTVqCjf90sjlGjoC8DR4_2B411ZwzCyYUIDIydqguJ43$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bxNF7CZVEFp0DL1eyWSI-q2mEvdEpUJBuYVryE4HyTVqCjf90sjlGjoC8DR4_2B411ZwzCyYUIDIyVtt4Wrj$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260330/7e51c616/attachment-0001.html>