[petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?

Matthew Knepley knepley at gmail.com
Wed Dec 1 17:55:01 CST 2021 

On Wed, Dec 1, 2021 at 6:12 PM Betrie, Getnet <gbetrie at anl.gov> wrote:

> Here is a specific example. I have 10 global edges and 10 global vertices.
> The vertices and edges represent neurons and synapses in the brain,
> respectively. Vertices 0-7  and 7-9 have excitation and depression effect
> on the target vertices. The first rank owns 5 edges (0-4) and 6 vertices (
> 0-4 local and 8 ghost). The second rank owns 5 edges (5-9) and 6 vertices
> (5-9 local and 4 ghost). Note that edges 4 (neuron 8 depresses neuron 4)
> and 8 (neuron 4 excite neuron 8) that share the same vertices on different
> ranks. The generated Plex is correct based on the cone and support outputs.
> However, the SF object is completely wrong as shown below.
>

I do not think it is wrong.


> The first rank supposed to have 10 roots and one leaf (vertex 8),
>

No. The number of roots is the _complete_ point space, so here that is [0,
11), and the number of roots is correct.
There are two leaf vertices, 9 and 10, mapping to vertices 5 and 9 on
process 1. There is no way for me to know
what vertices are supposed to be shared in your model.

   Matt


> and the second rank suppose to have the same number of roots and leaf
> (vertex 4) as the first rank. My guess is something went wrong in *sfPoint
> *when determining the shared local vertices. I can provide more outputs
> from DMPlex if it helps, please let me know.
>
> DM Object: 2 MPI processes
>   type: network
>   NSubnets: 1; NEdges: 10; NVertices: 10; NSharedVertices: 0.
>   [0] nEdges: 5; nVertices: 6; nSharedVertices: 0
>      Subnet 0: nEdges 5, nVertices(include shared vertices) 6
>        edge 0: 4 ----> 0
>        edge 1: 4 ----> 1
>        edge 2: 4 ----> 2
>        edge 3: 4 ----> 3
>        edge 4: 8 ----> 4
>   [1] nEdges: 5; nVertices: 6; nSharedVertices: 0
>      Subnet 0: nEdges 5, nVertices(include shared vertices) 6
>        edge 5: 4 ----> 5
>        edge 6: 4 ----> 6
>        edge 7: 4 ----> 7
>        edge 8: 4 ----> 8
>        edge 9: 4 ----> 9
>
> Cones:
> [0] Max cone size: 2
> [0]: 0 <---- 9 (0)
> [0]: 0 <---- 5 (0)
> [0]: 1 <---- 9 (0)
> [0]: 1 <---- 6 (0)
> [0]: 2 <---- 9 (0)
> [0]: 2 <---- 7 (0)
> [0]: 3 <---- 9 (0)
> [0]: 3 <---- 8 (0)
> [0]: 4 <---- 10 (0)
> [0]: 4 <---- 9 (0)
> [1] Max cone size: 2
> [1]: 0 <---- 5 (0)
> [1]: 0 <---- 6 (0)
> [1]: 1 <---- 5 (0)
> [1]: 1 <---- 7 (0)
> [1]: 2 <---- 5 (0)
> [1]: 2 <---- 8 (0)
> [1]: 3 <---- 5 (0)
> [1]: 3 <---- 9 (0)
> [1]: 4 <---- 5 (0)
> [1]: 4 <---- 10 (0)
>
> PetscSF Object: 2 MPI processes
>   type: basic
>   [0] Number of roots=11, leaves=2, remote ranks=1
>   [0] 9 <- (1,5)
>   [0] 10 <- (1,9)
>   [1] Number of roots=11, leaves=0, remote ranks=0
>   [0] Roots referenced by my leaves, by rank
>   [0] 1: 2 edges
>   [0]    9 <- 5
>   [0]    10 <- 9
>   [1] Roots referenced by my leaves, by rank
>   MultiSF sort=rank-order
>
> ------------------------------
> *From:* Abhyankar, Shrirang G <shrirang.abhyankar at pnnl.gov>
> *Sent:* Wednesday, December 1, 2021 4:17 PM
> *To:* Knepley, Matthew G (VISIT) <knepley at gmail.com>; Barry Smith <
> bsmith at petsc.dev>
> *Cc:* petsc-dev <petsc-dev at mcs.anl.gov>; Zhang, Hong <hzhang at mcs.anl.gov>;
> Betrie, Getnet <gbetrie at anl.gov>
> *Subject:* Re: DMPLEX cannot support two different edges for the same two
> vertices, hence DMPLEX cannot?
>
>
> “You can certainly have many fields on a given edge, but I don't know
> what it would mean to have two edges since no topological query could tell
> the difference.”
>
>
>
> The two edges in a power grid represent two parallel power lines that are
> connected between two locations (vertices). There are line ids (stored in
> the component data) to distinguish the two lines.
>
>
>
> Thanks,
>
> Shri
>
>
>
>
>
> *From: *Matthew Knepley <knepley at gmail.com>
> *Date: *Wednesday, December 1, 2021 at 4:07 PM
> *To: *Barry Smith <bsmith at petsc.dev>
> *Cc: *"Abhyankar, Shrirang G" <shrirang.abhyankar at pnnl.gov>, PETSc
> Development <petsc-dev at mcs.anl.gov>, "Zhang, Hong" <hzhang at mcs.anl.gov>,
> Getnet Betrie <gbetrie at anl.gov>
> *Subject: *Re: DMPLEX cannot support two different edges for the same two
> vertices, hence DMPLEX cannot?
>
>
>
> On Wed, Dec 1, 2021 at 4:51 PM Barry Smith <bsmith at petsc.dev> wrote:
>
>
>
>   It may not happen with one rank. Or it may.
>
>
>
>   With more than one rank one may run a code and think it is working
> correctly but that does not mean the PetscSF is correct. It is generating
> "extra" degrees of freedom for these vertices each associated with a
> different edge and not properly connecting them. You can tell by doing a
> full DMView on the dmplex and PetscSF and seeing more roots than there
> should be listed. Hence the numerics will be wrong even though it does not
> crash or complain.
>
>
>
> Okay, I think we need specific tests to see what is going on.
>
>
>
> Theoretically, having multiple edges between vertices does not make sense
> since the edge _means_ (v_i, v_j) topologically. You can certainly have
> many fields on a given edge, but I don't know what it would mean to have
> two edges since no topological query could tell the difference.
>
>
>
> However, you can certainly tell Plex anything you want to tell it.
> DMPlexBuildFromCellListParallel() gets only cells and vertices, so you are
> saying that there is a duplicate cell. That is allowed and should work
> properly, EXCEPT if you wanted to identify a cell only by its cone (then
> you could not tell them apart). I do this for faces, but I did not think I
> ever did it for cells.
>
>
>
>   Thanks,
>
>
>
>      Matt
>
>
>
> On Dec 1, 2021, at 4:37 PM, Abhyankar, Shrirang G <
> shrirang.abhyankar at pnnl.gov> wrote:
>
>
>
> Barry,
>
>
>
> “Is there anything we can do to support having multiple edges between the
> same two vertices”
>
>
>
> Some of my power grid datasets have multiple edges between the same two
> vertices and I’ve not faced an issue with DMNetwork. However, all the data
> was read on rank 0 only (and then distributed).
>
>
>
> Maybe the issue is with the edges being passed on different ranks?
>
>
>
> Thanks,
>
> Shri
>
> *From: *Barry Smith <bsmith at petsc.dev>
> *Date: *Wednesday, December 1, 2021 at 3:19 PM
> *To: *"Knepley, Matthew G (VISIT)" <knepley at gmail.com>, PETSc Development
> <petsc-dev at mcs.anl.gov>, "Abhyankar, Shrirang G" <
> shrirang.abhyankar at pnnl.gov>, "Zhang, Hong" <hzhang at mcs.anl.gov>, Getnet
> Betrie <gbetrie at anl.gov>
> *Subject: *DMPLEX cannot support two different edges for the same two
> vertices, hence DMPLEX cannot?
>
>
>
> Check twice before you click! This email originated from outside PNNL.
>
>
>
>
>
>    Matt,
>
>
>
>      If DMPlexBuildFromCellListParallel() is called with two edges that
> have the same two vertices what will happen? It looks like it ends up with
> an incorrect PetscSF if the two edges are passed on different ranks. Hence
> the DMPLEX is not valid and produces garbage.
>
>
>
>      Neurons can be connected to themselves which seems to be breaking
> DMPLEX and hence DMNETWORK.
>
>
>
>      Is there anything we can do to support having multiple edges between
> the same two vertices? If not is there a way we can have
> DMPlexBuildFromCellListParallel() generate an error automatically if there
> are such extra edges in the input data.
>
>
>
>    Thanks
>
>
>
>   Barry
>
>
>
> In this work, the neurons are represented by vertices in the network and
> each synapse is a graph edge.
>
>
>
>
>
>
> --
>
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>
> https://www.cse.buffalo.edu/~knepley/
> <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cse.buffalo.edu%2F~knepley%2F&data=04%7C01%7Cshrirang.abhyankar%40pnnl.gov%7Ce56f4d6ffa10474f3c7508d9b516feb5%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C637739932634554446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WwSfLYl2hPyHwrfkJ1SuBmp6l88d6xJulwBPBvu%2BbK0%3D&reserved=0>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20211201/65260b6e/attachment-0001.html>

More information about the petsc-dev mailing list