[petsc-users] DMSwarm Initialization on a DMPlex (PIC scheme)

[Miguel Molinos]

Thank you Matt

Indeed the right way was to use the rank property of the SF. 

Thanks, 
Miguel

  PetscCall(
      DMLocatePoints(FE_Mesh, coordinates, DM_POINTLOCATION_NONE, &cellSF));
  PetscCall(PetscSFGetGraph(cellSF, &nroots, NULL, NULL, &LA_sfcell));

  PetscCall(VecGetArray(coordinates, &coordinates_ptr));

  // Note: mean_q_ptr should be filled based on n_atoms_local increment
  for (PetscInt site_i = 0; site_i < n_atoms; site_i++) {
    // Check if the point is found in the local process
    if (LA_sfcell[site_i].rank == rank_MPI) {
      idx_ptr[n_atoms_local] = site_i;
      for (int alpha = 0; alpha < dim; alpha++) {
        mean_q_ptr[n_atoms_local * dim + alpha] =
            Simulation_file.mean_q[site_i * dim + alpha];
      }
      n_atoms_local++;
    }
  }


> On Mar 30, 2026, at 2:50 AM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Sun, Mar 29, 2026 at 6:25 PM Miguel Molinos <m.molinos at upm.es <mailto:m.molinos at upm.es>> wrote:
>> Dear all,
>> 
>> I am experiencing an issue where particles created with DMSwarm are being increased across MPI ranks when initializing a DMSwarm over a periodic DMPlex. I follow this procedure on every rank:
>> 
>> Create a sequential vector with the global coordinates of all particles (18,000 atoms).
>> 
>> Call DMLocatePoints(FE_Mesh, coordinates, DM_POINTLOCATION_NONE, &cellSF) to find which particles belong to the local DMPlex partition. Then call PetscSFGetGraph.
>> 
>> Iterate through the results and add any particle with a valid cell index (index != DMLOCATEPOINT_POINT_NOT_FOUND) to the local DMSwarm.
>> 
>> The sum of n_atoms_local across all ranks gives 18,740 instead of the expected 18,000. It seems that particles located exactly on periodic boundaries or MPI interfaces are being identified as "found" by multiple ranks simultaneously. Does DMLocatePoints return points found in ghost/halo cells by default when the DMPlex is periodic or distributed with overlap?
>> 
> I do not think this should happen at periodic boundaries, but definitely if you have cell overlap it will happen. It iterates over all local cells to check particle location. You could discard those points by checking if the cell is in the point SF in Step 3.
> 
>   Thanks,
> 
>       Matt 
>> Thanks,
>> 
>> Miguel
>> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dhM8KIoPRZB22Da7-tZeGcTAKt33F9B8fGduBARq61MGINsttNyOQ0iz4oOuKSstx8O9_w-Ukkrip9fFSmO136sx$  <https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!D9dNQwwGXtA!X58muDkArsK-iH2YdyGstzlwOwzuxXNE308JQOmlP5pY7V8oHcgLbmf1Qiazmqv1IdaROicSzLQR_o8$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260330/030d05bd/attachment.html>