<div dir="ltr"><div dir="ltr">On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G <<a href="mailto:shrirang.abhyankar@pnnl.gov">shrirang.abhyankar@pnnl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_-2812463541188624746WordSection1">
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif">“</span>You can certainly have many fields on a given edge, but I don't know what it would mean to have two edges since no topological query could tell the difference.”<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The two edges in a power grid represent two parallel power lines that are connected between two locations (vertices). There are line ids (stored in the component data) to distinguish the two lines.</p></div></div></blockquote><div><br></div><div>Yes, so you can tell the difference in the function space (since difference current passes down each one), but _topologically_ you cannot. If you put duplicate cells in, then</div><div>some topological queries will give unexpected results, like the join of the two vertices.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US" style="overflow-wrap: break-word;"><div class="gmail-m_-2812463541188624746WordSection1"><p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt"> </span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif">Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif">Shri <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif"><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal" style="margin-left:0.5in"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Wednesday, December 1, 2021 at 4:07 PM<br>
<b>To: </b>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>><br>
<b>Cc: </b>"Abhyankar, Shrirang G" <<a href="mailto:shrirang.abhyankar@pnnl.gov" target="_blank">shrirang.abhyankar@pnnl.gov</a>>, PETSc Development <<a href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>>, "Zhang, Hong" <<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>>, Getnet Betrie <<a href="mailto:gbetrie@anl.gov" target="_blank">gbetrie@anl.gov</a>><br>
<b>Subject: </b>Re: DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">On Wed, Dec 1, 2021 at 4:51 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<u></u><u></u></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> It may not happen with one rank. Or it may.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-left:0.5in"> With more than one rank one may run a code and think it is working correctly but that does not mean the PetscSF is correct. It is generating "extra" degrees of freedom for these vertices each associated with a
different edge and not properly connecting them. You can tell by doing a full DMView on the dmplex and PetscSF and seeing more roots than there should be listed. Hence the numerics will be wrong even though it does not crash or complain.<u></u><u></u></p>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Okay, I think we need specific tests to see what is going on.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Theoretically, having multiple edges between vertices does not make sense since the edge _means_ (v_i, v_j) topologically. You can certainly have many fields on a given edge, but I don't know what it would mean
to have two edges since no topological query could tell the difference.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">However, you can certainly tell Plex anything you want to tell it. DMPlexBuildFromCellListParallel() gets only cells and vertices, so you are saying that there is a duplicate cell. That is allowed and should work
properly, EXCEPT if you wanted to identify a cell only by its cone (then you could not tell them apart). I do this for faces, but I did not think I ever did it for cells.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> Matt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal" style="margin-left:0.5in">On Dec 1, 2021, at 4:37 PM, Abhyankar, Shrirang G <<a href="mailto:shrirang.abhyankar@pnnl.gov" target="_blank">shrirang.abhyankar@pnnl.gov</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif">Barry,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif">“</span>Is there anything we can do to support having multiple edges between the same two vertices”<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Some of my power grid datasets have multiple edges between the same two vertices and I’ve not faced an issue with DMNetwork. However, all the data was read on rank 0 only (and then distributed).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Maybe the issue is with the edges being passed on different ranks?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif"> </span><u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif">Thanks,</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:12pt;font-family:"Times New Roman",serif">Shri </span><u></u><u></u></p>
</div>
</div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"><b><span style="font-size:12pt">From: </span></b><span style="font-size:12pt">Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>><br>
<b>Date: </b>Wednesday, December 1, 2021 at 3:19 PM<br>
<b>To: </b>"Knepley, Matthew G (VISIT)" <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>, PETSc Development <<a href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>>, "Abhyankar, Shrirang G" <<a href="mailto:shrirang.abhyankar@pnnl.gov" target="_blank">shrirang.abhyankar@pnnl.gov</a>>,
"Zhang, Hong" <<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>>, Getnet Betrie <<a href="mailto:gbetrie@anl.gov" target="_blank">gbetrie@anl.gov</a>><br>
<b>Subject: </b>DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?</span><u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in">Check twice before you click! This email originated from outside PNNL.<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> Matt,<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> If DMPlexBuildFromCellListParallel() is called with two edges that have the same two vertices what will happen? It looks like it ends up with an incorrect PetscSF if the two edges are passed on different ranks.
Hence the DMPLEX is not valid and produces garbage.<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> Neurons can be connected to themselves which seems to be breaking DMPLEX and hence DMNETWORK.<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> Is there anything we can do to support having multiple edges between the same two vertices? If not is there a way we can have DMPlexBuildFromCellListParallel() generate an error automatically if there are such
extra edges in the input data.<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> Thanks<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> Barry<u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in"> <u></u><u></u></p>
</div>
</div>
<div>
<div style="margin-left:0.5in">
<p class="MsoNormal" style="margin-left:0.5in">In this work, the neurons are represented by vertices in the network and each synapse is a graph edge.<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:0.5in"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-left:0.5in">-- <u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><a href="https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cse.buffalo.edu%2F~knepley%2F&data=04%7C01%7Cshrirang.abhyankar%40pnnl.gov%7Ce56f4d6ffa10474f3c7508d9b516feb5%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C637739932634554446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WwSfLYl2hPyHwrfkJ1SuBmp6l88d6xJulwBPBvu%2BbK0%3D&reserved=0" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>