[petsc-users] Ghost vector with MPIKOKKOS under DMSwarm

[MIGUEL MOLINOS PEREZ]

Understood!

I think I can make it work because the way I create ghost particles in my code is with:
(i)DMSwarmAddNPoints,
(ii) two auxiliar variables named idx_ghost and rank_ghost,
(iii) and DMSWARM_MIGRATE_BASIC for the migration.

In summary, shared particles are indeed at the end of the Vec.

Actually you helped me with this hack months ago :-).

I think I can use the code you mentioned as an inspiration… let's see!

Miguel

On Jan 19, 2026, at 7:26 PM, Matthew Knepley <knepley at gmail.com> wrote:

The code is the right place to look. For example

  https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/swarm/swarm.c?ref_type=heads*L307__;Iw!!G_uCfscf7eWS!dB-a-Zi9Mzr_7h5tsaTEWhXfpl8M-dAWMu3CsQ4axsXQNCXMO3XGub6G8G8OaSjDdg5zakXL6G1r8et7--tHHA$ 

So the Vec will get the right type. This means that when the initial data comes in on the host, it will be copied to the device when we execute an operation.

We (really Joe) are working on an update that allows swarm data to be held on the device.

  Thanks,

     Matt

On Mon, Jan 19, 2026 at 1:22 PM MIGUEL MOLINOS PEREZ <mmolinos at us.es<mailto:mmolinos at us.es>> wrote:
I’ll check the documentation more carefully to see if I can find this.

Thanks,
Miguel

On Jan 19, 2026, at 6:42 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:

On Mon, Jan 19, 2026 at 12:19 PM MIGUEL MOLINOS PEREZ <mmolinos at us.es<mailto:mmolinos at us.es>> wrote:
I don’t think so because DMSwarm data is allocated on host. This is right Matt?

I believe there is code in there to move it if you give a device type.

  Thanks,

     Matt

Thanks,
Miguel

On Jan 19, 2026, at 5:44 PM, Junchao Zhang <junchao.zhang at gmail.com<mailto:junchao.zhang at gmail.com>> wrote:

With VecCreateGhostWithArray, do you already have the array ready on device?

--Junchao Zhang


On Mon, Jan 19, 2026 at 6:56 AM MIGUEL MOLINOS PEREZ <mmolinos at us.es<mailto:mmolinos at us.es>> wrote:

Dear all,

I have a question about using DMSwarm data with Kokkos-enabled vectors.

My particle data (including ghost particles) are stored in a DMSwarm and for solver purposes I generate PETSc vectors using VecCreateGhostWithArray. I would like to use VecMPIKOKKOS for MPI+GPU computations. Am I correct that DMSwarm field memory cannot be directly wrapped into a VecMPIKOKKOS using VecCreateGhostWithArray or VecPlaceArray? Any idea on how to proceed?

Thanks in advance.

Best,

Miguel



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dB-a-Zi9Mzr_7h5tsaTEWhXfpl8M-dAWMu3CsQ4axsXQNCXMO3XGub6G8G8OaSjDdg5zakXL6G1r8etBf76ZFw$ <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dB-a-Zi9Mzr_7h5tsaTEWhXfpl8M-dAWMu3CsQ4axsXQNCXMO3XGub6G8G8OaSjDdg5zakXL6G1r8evBevKFTQ$ >



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dB-a-Zi9Mzr_7h5tsaTEWhXfpl8M-dAWMu3CsQ4axsXQNCXMO3XGub6G8G8OaSjDdg5zakXL6G1r8etBf76ZFw$ <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dB-a-Zi9Mzr_7h5tsaTEWhXfpl8M-dAWMu3CsQ4axsXQNCXMO3XGub6G8G8OaSjDdg5zakXL6G1r8evBevKFTQ$ >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260119/3e04a8d2/attachment-0001.html>