[petsc-users] Right DM for a particle network

Wed Jul 31 09:08:35 CDT 2024

Just a thought, but perhaps he may want to use just sparse matrices, AIJ.
He Manages the connectivity And we manage ghost values.

On Wed, Jul 31, 2024 at 6:25 AM Matthew Knepley <knepley at gmail.com> wrote:

> On Tue, Jul 30, 2024 at 11:32 PM Marco Seiz <marco at kit.ac.jp> wrote:
>
>> Hello,
>>
>> maybe to clarify a bit further: I'd essentially like to solve heat
>> transport between particles only, without solving the transport on my voxel
>> mesh since there's a large scale difference between the voxel size and the
>> particle size and heat transport should be fast enough that voxel
>> resolution is unnecessary. Basically a discrete element method just for
>> heat transport. The whole motion/size change part is handled separately on
>> the voxel mesh.
>> Based on the connectivity, I can make a graph (attached an example from a
>> 3D case, for description see [1]) and on each vertex (particle) of the
>> graph I want to account for source terms and conduction along the edges.
>> What I'd like to avoid is managing the exchange for non-locally owned
>> vertices during the solve (e.g. for RHS evaluation) myself but rather have
>> the DM do it with DMGlobalToLocal() and friends. Thinking a bit further,
>> I'd probably also want to associate some data with the edges since that
>> will enter the conduction term but stays constant during a solve (think
>> contact area between particles).
>>
>> Looking over the DMSwarm examples, the coupling between particles is via
>> the background mesh, so e.g. I can't say "loop over all local particles and
>> for each particle and its neighbours do X". I could use the projection part
>> to dump the the source terms from the particles to a coarser background
>> mesh but for the conduction term I don't see how I could get a good
>> approximation of the contact area on the background mesh without having a
>> mesh at a similar resolution as I already have, kinda destroying the
>> purpose of the whole thing.
>>
>
> The point I was trying to make in my previous message is that DMSwarm does
> not require a background mesh. The examples use one because that is what we
> use to evaluate particle grouping. However, you have an independent way to
> do this, so you do not need it.
>
> Second, the issue of replicated particles. DMSwarmMigrate allows you to
> replicate particles, using the input flag. Of course, you would have to
> manage removing particles you no longer want.
>
>   Thanks,
>
>      Matt
>
>
>> [1] Points represent particles, black lines are edges, with the color
>> indicating which worker "owns" the particle, with 3 workers being used and
>> only a fraction of edges/vertices being displayed to keep it somewhat tidy.
>> The position of the points corresponds to the particles' x-y position, with
>> the z position being ignored. Particle ownership isn't done via looking
>> where the particle is on the voxel grid, but rather by dividing the array
>> of particle indices into subarrays, so e.g. particles [0-n/3) go to the
>> first worker and so on. Since my particles can span multiple workers on the
>> voxel grid this makes it much easier to update edge information with
>> one-sided communication.  As you can see the "mesh" is quite irregular with
>> no nice boundary existing for connected particles owned by different
>> workers.
>>
>> Best regards,
>> Marco
>>
>> On 30.07.24 22:56, Mark Adams wrote:
>> > * they do have a  vocal mesh, so perhaps They want DM Plex.
>> >
>> > * they want ghost particle communication, that also might want a mesh
>> >
>> > * DM swarm does not have a notion of ghost particle, as far as I know,
>> but it could use one
>> >
>> > On Tue, Jul 30, 2024 at 7:58 AM Matthew Knepley <knepley at gmail.com
>> <mailto:knepley at gmail.com>> wrote:
>> >
>> >     On Tue, Jul 30, 2024 at 12: 24 AM Marco Seiz <marco@ kit. ac. jp>
>> wrote: Hello, I'd like to solve transient heat transport at a particle
>> scale using TS, with the per-particle equation being something like dT_i /
>> dt = (S(T_i) + sum(F(T_j,
>> >     ZjQcmQRYFpfptBannerStart
>> >     __
>> >     This Message Is From an External Sender
>> >     This message came from outside your organization.
>> >
>> >     __
>> >     ZjQcmQRYFpfptBannerEnd
>> >     On Tue, Jul 30, 2024 at 12:24 AM Marco Seiz <marco at kit.ac.jp
>> <mailto:marco at kit.ac.jp>> wrote:
>> >
>> >         __
>> >         Hello, I'd like to solve transient heat transport at a particle
>> scale using TS, with the per-particle equation being something like dT_i /
>> dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i)) with a nonlinear source
>> term S and a conduction term
>> >         ZjQcmQRYFpfptBannerStart
>> >         __
>> >         This Message Is From an External Sender
>> >         This message came from outside your organization.
>> >
>> >         __
>> >         ZjQcmQRYFpfptBannerEnd
>> >
>> >         Hello,
>> >
>> >         I'd like to solve transient heat transport at a particle scale
>> using TS, with the per-particle equation being something like
>> >
>> >         dT_i / dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i))
>> >
>> >         with a nonlinear source term S and a conduction term F. The
>> particles can move, deform and grow/shrink/vanish on a voxel grid, but for
>> the temperature a particle-scale resolution should be sufficient. The
>> particles' connectivity will change during the simulation, but is assumed
>> constant during a single timestep. I have a data structure tracking the
>> particles' connectivity, so I can say which particles should conduct heat
>> to each other. I exploit symmetry and so only save the "forward" edges, so
>> e.g. for touching particles 1->2->3, I only store [[2], [3], []], from
>> which the full list [[2], [1, 3], [2]] could be reconstructed but which I'd
>> like to avoid. In parallel each worker would own some of the particle data,
>> so e.g. for the 1->2->3 example and 2 workers, worker 0 could own [[2]] and
>> worker 1 [[3],[]].
>> >
>> >         Looking over the DM variants, either DMNetwork or some manual
>> mesh build with DMPlex seem suited for this. I'd especially like it if the
>> adjacency information is handled by the DM automagically based on the edges
>> so I don't have to deal with ghost particle communication myself. I already
>> tried something basic with DMNetwork, though for some reason the offsets I
>> get from DMNetworkGetGlobalVecOffset() are larger than the actual network.
>> I've attached what I have so far but comparing to e.g.
>> src/snes/tutorials/network/ex1.c I don't see what I'm doing wrong if I
>> don't need data at the edges. I might not be seeing the trees for the
>> forest though. The output with -dmnetwork_view looks reasonable to me. Any
>> help in fixing this approach, or if it would seem suitable pointers to
>> using DMPlex for this problem, would be appreciated.
>> >
>> >     To me, this sounds like you should built it with DMSwarm. Why?
>> >
>> >     1) We only have vertices and edges, so a mesh does not buy us
>> anything.
>> >
>> >     2) You are managing the parallel particle connectivity, so DMPlex
>> topology is not buying us anything. Unless I am misunderstanding.
>> >
>> >     3) DMNetwork has a lot of support for vertices with different
>> characteristics. Your particles all have the same attributes, so this is
>> unnecessary.
>> >
>> >     How would you set this up?
>> >
>> >     1) Declare all particle attributes. There are many Swarm examples,
>> but say ex6 which simulates particles moving under a central force.
>> >
>> >     2) That example decides when to move particles using a parallel
>> background mesh. However, you know which particles you want to move,
>> >          so you just change the _rank_ field to the new rank and call
>> DMSwarmMigrate() with migration type _basic_.
>> >
>> >     It should be straightforward to setup a tiny example moving around
>> a few particles to see if it does everything you want.
>> >
>> >        Thanks,
>> >
>> >          Matt
>> >
>> >
>> >         Best regards,
>> >         Marco
>> >
>> >     --
>> >     What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> >     -- Norbert Wiener
>> >
>> >     https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZJr2_J0yDqE986DrPL02S_TaBzfsRouKOJpjlNaYGxRPDZjvsFmDXuM54NAtwLM9H4zC3A2fiCmrGuv4SoW9qjg$  <
>> https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bLVHnoUGooYpdfGD8zNQrHTY2ln70W082hEc6pG7vdjA2fCvs77tcI9d7QOA0i_FjGK1of3nNOKXCE-4Rwb0$
>> >
>> >
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZJr2_J0yDqE986DrPL02S_TaBzfsRouKOJpjlNaYGxRPDZjvsFmDXuM54NAtwLM9H4zC3A2fiCmrGuv4SoW9qjg$ 
> <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZJr2_J0yDqE986DrPL02S_TaBzfsRouKOJpjlNaYGxRPDZjvsFmDXuM54NAtwLM9H4zC3A2fiCmrGuv4YUBXTig$ >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240731/d15d6e24/attachment-0001.html>