<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div><div class=""> It may not happen with one rank. Or it may.</div><div class=""><br class=""></div> With more than one rank one may run a code and think it is working correctly but that does not mean the PetscSF is correct. It is generating "extra" degrees of freedom for these vertices each associated with a different edge and not properly connecting them. You can tell by doing a full DMView on the dmplex and PetscSF and seeing more roots than there should be listed. Hence the numerics will be wrong even though it does not crash or complain.<div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Dec 1, 2021, at 4:37 PM, Abhyankar, Shrirang G <<a href="mailto:shrirang.abhyankar@pnnl.gov" class="">shrirang.abhyankar@pnnl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">Barry,<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">“</span>Is there anything we can do to support having multiple edges between the same two vertices”<o:p class=""></o:p></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Some of my power grid datasets have multiple edges between the same two vertices and I’ve not faced an issue with DMNetwork. However, all the data was read on rank 0 only (and then distributed).<o:p class=""></o:p></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Maybe the issue is with the edges being passed on different ranks?<span style="font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></span></div><div class=""><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">Thanks,<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 12pt; font-family: "Times New Roman", serif;" class="">Shri <o:p class=""></o:p></span></div></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0in 0in;" class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 12pt;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;" class="">Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>><br class=""><b class="">Date:<span class="Apple-converted-space"> </span></b>Wednesday, December 1, 2021 at 3:19 PM<br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>"Knepley, Matthew G (VISIT)" <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>>, PETSc Development <<a href="mailto:petsc-dev@mcs.anl.gov" class="">petsc-dev@mcs.anl.gov</a>>, "Abhyankar, Shrirang G" <<a href="mailto:shrirang.abhyankar@pnnl.gov" class="">shrirang.abhyankar@pnnl.gov</a>>, "Zhang, Hong" <<a href="mailto:hzhang@mcs.anl.gov" class="">hzhang@mcs.anl.gov</a>>, Getnet Betrie <<a href="mailto:gbetrie@anl.gov" class="">gbetrie@anl.gov</a>><br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Check twice before you click! This email originated from outside PNNL.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> Matt,<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> If DMPlexBuildFromCellListParallel() is called with two edges that have the same two vertices what will happen? It looks like it ends up with an incorrect PetscSF if the two edges are passed on different ranks. Hence the DMPLEX is not valid and produces garbage.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> Neurons can be connected to themselves which seems to be breaking DMPLEX and hence DMNETWORK.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> Is there anything we can do to support having multiple edges between the same two vertices? If not is there a way we can have DMPlexBuildFromCellListParallel() generate an error automatically if there are such extra edges in the input data.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> Thanks<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> Barry<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0in 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">In this work, the neurons are represented by vertices in the network and each synapse is a graph edge.</div></div></div></div></blockquote></div><br class=""></div></body></html>