[petsc-users] Right DM for a particle network

Matthew Knepley knepley at gmail.com
Wed Jul 31 05:24:56 CDT 2024 

On Tue, Jul 30, 2024 at 11:32 PM Marco Seiz <marco at kit.ac.jp> wrote:

> Hello,
>
> maybe to clarify a bit further: I'd essentially like to solve heat
> transport between particles only, without solving the transport on my voxel
> mesh since there's a large scale difference between the voxel size and the
> particle size and heat transport should be fast enough that voxel
> resolution is unnecessary. Basically a discrete element method just for
> heat transport. The whole motion/size change part is handled separately on
> the voxel mesh.
> Based on the connectivity, I can make a graph (attached an example from a
> 3D case, for description see [1]) and on each vertex (particle) of the
> graph I want to account for source terms and conduction along the edges.
> What I'd like to avoid is managing the exchange for non-locally owned
> vertices during the solve (e.g. for RHS evaluation) myself but rather have
> the DM do it with DMGlobalToLocal() and friends. Thinking a bit further,
> I'd probably also want to associate some data with the edges since that
> will enter the conduction term but stays constant during a solve (think
> contact area between particles).
>
> Looking over the DMSwarm examples, the coupling between particles is via
> the background mesh, so e.g. I can't say "loop over all local particles and
> for each particle and its neighbours do X". I could use the projection part
> to dump the the source terms from the particles to a coarser background
> mesh but for the conduction term I don't see how I could get a good
> approximation of the contact area on the background mesh without having a
> mesh at a similar resolution as I already have, kinda destroying the
> purpose of the whole thing.
>

The point I was trying to make in my previous message is that DMSwarm does
not require a background mesh. The examples use one because that is what we
use to evaluate particle grouping. However, you have an independent way to
do this, so you do not need it.

Second, the issue of replicated particles. DMSwarmMigrate allows you to
replicate particles, using the input flag. Of course, you would have to
manage removing particles you no longer want.

  Thanks,

     Matt


> [1] Points represent particles, black lines are edges, with the color
> indicating which worker "owns" the particle, with 3 workers being used and
> only a fraction of edges/vertices being displayed to keep it somewhat tidy.
> The position of the points corresponds to the particles' x-y position, with
> the z position being ignored. Particle ownership isn't done via looking
> where the particle is on the voxel grid, but rather by dividing the array
> of particle indices into subarrays, so e.g. particles [0-n/3) go to the
> first worker and so on. Since my particles can span multiple workers on the
> voxel grid this makes it much easier to update edge information with
> one-sided communication.  As you can see the "mesh" is quite irregular with
> no nice boundary existing for connected particles owned by different
> workers.
>
> Best regards,
> Marco
>
> On 30.07.24 22:56, Mark Adams wrote:
> > * they do have a  vocal mesh, so perhaps They want DM Plex.
> >
> > * they want ghost particle communication, that also might want a mesh
> >
> > * DM swarm does not have a notion of ghost particle, as far as I know,
> but it could use one
> >
> > On Tue, Jul 30, 2024 at 7:58 AM Matthew Knepley <knepley at gmail.com
> <mailto:knepley at gmail.com>> wrote:
> >
> >     On Tue, Jul 30, 2024 at 12: 24 AM Marco Seiz <marco@ kit. ac. jp>
> wrote: Hello, I'd like to solve transient heat transport at a particle
> scale using TS, with the per-particle equation being something like dT_i /
> dt = (S(T_i) + sum(F(T_j,
> >     ZjQcmQRYFpfptBannerStart
> >     __
> >     This Message Is From an External Sender
> >     This message came from outside your organization.
> >
> >     __
> >     ZjQcmQRYFpfptBannerEnd
> >     On Tue, Jul 30, 2024 at 12:24 AM Marco Seiz <marco at kit.ac.jp
> <mailto:marco at kit.ac.jp>> wrote:
> >
> >         __
> >         Hello, I'd like to solve transient heat transport at a particle
> scale using TS, with the per-particle equation being something like dT_i /
> dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i)) with a nonlinear source
> term S and a conduction term
> >         ZjQcmQRYFpfptBannerStart
> >         __
> >         This Message Is From an External Sender
> >         This message came from outside your organization.
> >
> >         __
> >         ZjQcmQRYFpfptBannerEnd
> >
> >         Hello,
> >
> >         I'd like to solve transient heat transport at a particle scale
> using TS, with the per-particle equation being something like
> >
> >         dT_i / dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i))
> >
> >         with a nonlinear source term S and a conduction term F. The
> particles can move, deform and grow/shrink/vanish on a voxel grid, but for
> the temperature a particle-scale resolution should be sufficient. The
> particles' connectivity will change during the simulation, but is assumed
> constant during a single timestep. I have a data structure tracking the
> particles' connectivity, so I can say which particles should conduct heat
> to each other. I exploit symmetry and so only save the "forward" edges, so
> e.g. for touching particles 1->2->3, I only store [[2], [3], []], from
> which the full list [[2], [1, 3], [2]] could be reconstructed but which I'd
> like to avoid. In parallel each worker would own some of the particle data,
> so e.g. for the 1->2->3 example and 2 workers, worker 0 could own [[2]] and
> worker 1 [[3],[]].
> >
> >         Looking over the DM variants, either DMNetwork or some manual
> mesh build with DMPlex seem suited for this. I'd especially like it if the
> adjacency information is handled by the DM automagically based on the edges
> so I don't have to deal with ghost particle communication myself. I already
> tried something basic with DMNetwork, though for some reason the offsets I
> get from DMNetworkGetGlobalVecOffset() are larger than the actual network.
> I've attached what I have so far but comparing to e.g.
> src/snes/tutorials/network/ex1.c I don't see what I'm doing wrong if I
> don't need data at the edges. I might not be seeing the trees for the
> forest though. The output with -dmnetwork_view looks reasonable to me. Any
> help in fixing this approach, or if it would seem suitable pointers to
> using DMPlex for this problem, would be appreciated.
> >
> >     To me, this sounds like you should built it with DMSwarm. Why?
> >
> >     1) We only have vertices and edges, so a mesh does not buy us
> anything.
> >
> >     2) You are managing the parallel particle connectivity, so DMPlex
> topology is not buying us anything. Unless I am misunderstanding.
> >
> >     3) DMNetwork has a lot of support for vertices with different
> characteristics. Your particles all have the same attributes, so this is
> unnecessary.
> >
> >     How would you set this up?
> >
> >     1) Declare all particle attributes. There are many Swarm examples,
> but say ex6 which simulates particles moving under a central force.
> >
> >     2) That example decides when to move particles using a parallel
> background mesh. However, you know which particles you want to move,
> >          so you just change the _rank_ field to the new rank and call
> DMSwarmMigrate() with migration type _basic_.
> >
> >     It should be straightforward to setup a tiny example moving around a
> few particles to see if it does everything you want.
> >
> >        Thanks,
> >
> >          Matt
> >
> >
> >         Best regards,
> >         Marco
> >
> >     --
> >     What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> >     -- Norbert Wiener
> >
> >     https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!c2gVsrwkHTL4PROAsy9Pu3vtrctvJ1pJIz8p7ZFKYBh8wzleus-q3_IRynQ6EzGWXU-JGCBltgv6ciTTLl3x$  <
> https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bLVHnoUGooYpdfGD8zNQrHTY2ln70W082hEc6pG7vdjA2fCvs77tcI9d7QOA0i_FjGK1of3nNOKXCE-4Rwb0$
> >
> >



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!c2gVsrwkHTL4PROAsy9Pu3vtrctvJ1pJIz8p7ZFKYBh8wzleus-q3_IRynQ6EzGWXU-JGCBltgv6ciTTLl3x$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!c2gVsrwkHTL4PROAsy9Pu3vtrctvJ1pJIz8p7ZFKYBh8wzleus-q3_IRynQ6EzGWXU-JGCBltgv6cnyJJj77$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240731/2a09cead/attachment.html>

More information about the petsc-users mailing list