[petsc-users] Right DM for a particle network

Matthew Knepley knepley at gmail.com
Wed Jul 31 09:28:14 CDT 2024 

On Wed, Jul 31, 2024 at 10:08 AM Mark Adams <mfadams at lbl.gov> wrote:

> Just a thought, but perhaps he may want to use just sparse matrices, AIJ.
> He Manages the connectivity And we manage ghost values.
>

He is reconfiguring the neighborhood (row) each time, so you would
essentially create a new matrix at each step with different sparsity. It
would definitely function, but I wonder if he would have enough local
information to construct the rows?

   Thanks,

      Matt


> On Wed, Jul 31, 2024 at 6:25 AM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Tue, Jul 30, 2024 at 11:32 PM Marco Seiz <marco at kit.ac.jp> wrote:
>>
>>> Hello,
>>>
>>> maybe to clarify a bit further: I'd essentially like to solve heat
>>> transport between particles only, without solving the transport on my voxel
>>> mesh since there's a large scale difference between the voxel size and the
>>> particle size and heat transport should be fast enough that voxel
>>> resolution is unnecessary. Basically a discrete element method just for
>>> heat transport. The whole motion/size change part is handled separately on
>>> the voxel mesh.
>>> Based on the connectivity, I can make a graph (attached an example from
>>> a 3D case, for description see [1]) and on each vertex (particle) of the
>>> graph I want to account for source terms and conduction along the edges.
>>> What I'd like to avoid is managing the exchange for non-locally owned
>>> vertices during the solve (e.g. for RHS evaluation) myself but rather have
>>> the DM do it with DMGlobalToLocal() and friends. Thinking a bit further,
>>> I'd probably also want to associate some data with the edges since that
>>> will enter the conduction term but stays constant during a solve (think
>>> contact area between particles).
>>>
>>> Looking over the DMSwarm examples, the coupling between particles is via
>>> the background mesh, so e.g. I can't say "loop over all local particles and
>>> for each particle and its neighbours do X". I could use the projection part
>>> to dump the the source terms from the particles to a coarser background
>>> mesh but for the conduction term I don't see how I could get a good
>>> approximation of the contact area on the background mesh without having a
>>> mesh at a similar resolution as I already have, kinda destroying the
>>> purpose of the whole thing.
>>>
>>
>> The point I was trying to make in my previous message is that DMSwarm
>> does not require a background mesh. The examples use one because that is
>> what we use to evaluate particle grouping. However, you have an independent
>> way to do this, so you do not need it.
>>
>> Second, the issue of replicated particles. DMSwarmMigrate allows you to
>> replicate particles, using the input flag. Of course, you would have to
>> manage removing particles you no longer want.
>>
>>   Thanks,
>>
>>      Matt
>>
>>
>>> [1] Points represent particles, black lines are edges, with the color
>>> indicating which worker "owns" the particle, with 3 workers being used and
>>> only a fraction of edges/vertices being displayed to keep it somewhat tidy.
>>> The position of the points corresponds to the particles' x-y position, with
>>> the z position being ignored. Particle ownership isn't done via looking
>>> where the particle is on the voxel grid, but rather by dividing the array
>>> of particle indices into subarrays, so e.g. particles [0-n/3) go to the
>>> first worker and so on. Since my particles can span multiple workers on the
>>> voxel grid this makes it much easier to update edge information with
>>> one-sided communication.  As you can see the "mesh" is quite irregular with
>>> no nice boundary existing for connected particles owned by different
>>> workers.
>>>
>>> Best regards,
>>> Marco
>>>
>>> On 30.07.24 22:56, Mark Adams wrote:
>>> > * they do have a  vocal mesh, so perhaps They want DM Plex.
>>> >
>>> > * they want ghost particle communication, that also might want a mesh
>>> >
>>> > * DM swarm does not have a notion of ghost particle, as far as I know,
>>> but it could use one
>>> >
>>> > On Tue, Jul 30, 2024 at 7:58 AM Matthew Knepley <knepley at gmail.com
>>> <mailto:knepley at gmail.com>> wrote:
>>> >
>>> >     On Tue, Jul 30, 2024 at 12: 24 AM Marco Seiz <marco@ kit. ac. jp>
>>> wrote: Hello, I'd like to solve transient heat transport at a particle
>>> scale using TS, with the per-particle equation being something like dT_i /
>>> dt = (S(T_i) + sum(F(T_j,
>>> >     ZjQcmQRYFpfptBannerStart
>>> >     __
>>> >     This Message Is From an External Sender
>>> >     This message came from outside your organization.
>>> >
>>> >     __
>>> >     ZjQcmQRYFpfptBannerEnd
>>> >     On Tue, Jul 30, 2024 at 12:24 AM Marco Seiz <marco at kit.ac.jp
>>> <mailto:marco at kit.ac.jp>> wrote:
>>> >
>>> >         __
>>> >         Hello, I'd like to solve transient heat transport at a
>>> particle scale using TS, with the per-particle equation being something
>>> like dT_i / dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i)) with a
>>> nonlinear source term S and a conduction term
>>> >         ZjQcmQRYFpfptBannerStart
>>> >         __
>>> >         This Message Is From an External Sender
>>> >         This message came from outside your organization.
>>> >
>>> >         __
>>> >         ZjQcmQRYFpfptBannerEnd
>>> >
>>> >         Hello,
>>> >
>>> >         I'd like to solve transient heat transport at a particle scale
>>> using TS, with the per-particle equation being something like
>>> >
>>> >         dT_i / dt = (S(T_i) + sum(F(T_j, T_i), j connecting to i))
>>> >
>>> >         with a nonlinear source term S and a conduction term F. The
>>> particles can move, deform and grow/shrink/vanish on a voxel grid, but for
>>> the temperature a particle-scale resolution should be sufficient. The
>>> particles' connectivity will change during the simulation, but is assumed
>>> constant during a single timestep. I have a data structure tracking the
>>> particles' connectivity, so I can say which particles should conduct heat
>>> to each other. I exploit symmetry and so only save the "forward" edges, so
>>> e.g. for touching particles 1->2->3, I only store [[2], [3], []], from
>>> which the full list [[2], [1, 3], [2]] could be reconstructed but which I'd
>>> like to avoid. In parallel each worker would own some of the particle data,
>>> so e.g. for the 1->2->3 example and 2 workers, worker 0 could own [[2]] and
>>> worker 1 [[3],[]].
>>> >
>>> >         Looking over the DM variants, either DMNetwork or some manual
>>> mesh build with DMPlex seem suited for this. I'd especially like it if the
>>> adjacency information is handled by the DM automagically based on the edges
>>> so I don't have to deal with ghost particle communication myself. I already
>>> tried something basic with DMNetwork, though for some reason the offsets I
>>> get from DMNetworkGetGlobalVecOffset() are larger than the actual network.
>>> I've attached what I have so far but comparing to e.g.
>>> src/snes/tutorials/network/ex1.c I don't see what I'm doing wrong if I
>>> don't need data at the edges. I might not be seeing the trees for the
>>> forest though. The output with -dmnetwork_view looks reasonable to me. Any
>>> help in fixing this approach, or if it would seem suitable pointers to
>>> using DMPlex for this problem, would be appreciated.
>>> >
>>> >     To me, this sounds like you should built it with DMSwarm. Why?
>>> >
>>> >     1) We only have vertices and edges, so a mesh does not buy us
>>> anything.
>>> >
>>> >     2) You are managing the parallel particle connectivity, so DMPlex
>>> topology is not buying us anything. Unless I am misunderstanding.
>>> >
>>> >     3) DMNetwork has a lot of support for vertices with different
>>> characteristics. Your particles all have the same attributes, so this is
>>> unnecessary.
>>> >
>>> >     How would you set this up?
>>> >
>>> >     1) Declare all particle attributes. There are many Swarm examples,
>>> but say ex6 which simulates particles moving under a central force.
>>> >
>>> >     2) That example decides when to move particles using a parallel
>>> background mesh. However, you know which particles you want to move,
>>> >          so you just change the _rank_ field to the new rank and call
>>> DMSwarmMigrate() with migration type _basic_.
>>> >
>>> >     It should be straightforward to setup a tiny example moving around
>>> a few particles to see if it does everything you want.
>>> >
>>> >        Thanks,
>>> >
>>> >          Matt
>>> >
>>> >
>>> >         Best regards,
>>> >         Marco
>>> >
>>> >     --
>>> >     What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> >     -- Norbert Wiener
>>> >
>>> >     https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eru_lbEwlvhK2Nbu6OsMVCn27QSZS67cnVfZgM-_OSyUCm_OPRqO4HSgfcDBzj2a9C4AC8cb_OVLajN1Jsba$  <
>>> https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bLVHnoUGooYpdfGD8zNQrHTY2ln70W082hEc6pG7vdjA2fCvs77tcI9d7QOA0i_FjGK1of3nNOKXCE-4Rwb0$
>>> >
>>> >
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eru_lbEwlvhK2Nbu6OsMVCn27QSZS67cnVfZgM-_OSyUCm_OPRqO4HSgfcDBzj2a9C4AC8cb_OVLajN1Jsba$ 
>> <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eru_lbEwlvhK2Nbu6OsMVCn27QSZS67cnVfZgM-_OSyUCm_OPRqO4HSgfcDBzj2a9C4AC8cb_OVLalTr8ibI$ >
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eru_lbEwlvhK2Nbu6OsMVCn27QSZS67cnVfZgM-_OSyUCm_OPRqO4HSgfcDBzj2a9C4AC8cb_OVLajN1Jsba$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eru_lbEwlvhK2Nbu6OsMVCn27QSZS67cnVfZgM-_OSyUCm_OPRqO4HSgfcDBzj2a9C4AC8cb_OVLalTr8ibI$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240731/3bb4fe90/attachment-0001.html>

More information about the petsc-users mailing list