<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Helvetica, sans-serif; ">
<div>I think you should call DMClone after partitioning (DMDistribute).</div>
<div><br>
</div>
<div>Shri</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Miguel Angel Salazar de Troya <<a href="mailto:salazardetroya@gmail.com">salazardetroya@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Mon, 23 Feb 2015 14:15:00 -0600<br>
<span style="font-weight:bold">To: </span>Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span>Shri <<a href="mailto:abhyshr@mcs.anl.gov">abhyshr@mcs.anl.gov</a>>, "<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [petsc-users] DMNetworkGetEdgeRange() in parallel<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div dir="ltr">
<div>Thanks a lot, the partition should be done before setting up the section, right?<br>
<br>
</div>
Miguel<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Feb 23, 2015 at 2:05 PM, Matthew Knepley <span dir="ltr">
<<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span class="">On Mon, Feb 23, 2015 at 1:40 PM, Miguel Angel Salazar de Troya
<span dir="ltr"><<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Wouldn't including the edge variables in the global vector make the code slower? I'm using the global vector in a TS, using one of the explicit RK schemes. The edge variables would not be updated in the RHSFunction evaluation. I only change the
edge variables in the TSUpdate. If the global vector had the edge variables, it would be a much larger vector, and all the vector operations performed by the TS would be slower. Although the vector F returned by the RHSFunction would be zero in the edge variable
components. I guess that being the vector sparse that would not be a problem.
<div><br>
</div>
<div>I think I'm more interested in the PetscSection approach because it might require less modifications in my code. However, I don't know how I could do this. Maybe something like this?</div>
<div><br>
</div>
<div>
<div>PetscSectionCreate(PETSC_COMM_WORLD, &s);</div>
<div>PetscSectionSetNumFields(s, 1);</div>
<div>PetscSectionSetFieldComponents(s, 0, 1);</div>
<div><br>
</div>
<div>// Now to set the chart, I pick the edge range</div>
<div>
<pre style="color:rgb(0,0,0)"><font face="arial,helvetica,sans-serif">DMNetworkGetEdgeRange(dm, & eStart, & eEnd</font></pre>
</div>
<div>PetscSectionSetChart(s, <span style="color: rgb(0, 0, 0); font-family: arial, helvetica, sans-serif; ">eStart</span>, <span style="color: rgb(0, 0, 0); font-family: arial, helvetica, sans-serif; ">eEnd</span>);<br>
</div>
<div><br>
</div>
<div>for(PetscInt e = <span style="color: rgb(0, 0, 0); font-family: arial, helvetica, sans-serif; ">eStart</span>; c < <span style="color: rgb(0, 0, 0); font-family: arial, helvetica, sans-serif; ">eEnd</span>; ++e) {<br>
</div>
<div> PetscSectionSetDof(s, e, 1);</div>
<div> PetscSectionSetFieldDof(s, e, 1, 1);</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>It should be PetscSectionSetFieldDof(s, e, 0, 1);</div>
<span class="">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>}</div>
<div>PetscSectionSetUp(s);</div>
</div>
<div><br>
</div>
<div>Now in the manual I see this:</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>First you want to do:</div>
<div><br>
</div>
<div> DMClone(dm, &dmEdge);</div>
<div><br>
</div>
<div>and then use dmEdge below.</div>
<span class="">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>DMSetDefaultSection(dm, s);</div>
<div>DMGetLocalVector(dm, &localVec);</div>
<div>DMGetGlobalVector(dm, &globalVec);</div>
</div>
<div><br>
</div>
<div>Setting up the default section in the DM would interfere with the section already set up with the variables in the vertices?</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>Yep, thats why you would use a clone.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div>
<div class="h5">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>Thanks a lot for your responses.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Feb 23, 2015 at 11:37 AM, Matthew Knepley <span dir="ltr">
<<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>On Mon, Feb 23, 2015 at 9:27 AM, Miguel Angel Salazar de Troya
<span dir="ltr"><<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">I'm iterating through local edges given in <span style="font-size:13px">DMNetworkGetEdgeRange(</span><span style="font-size:13px">). For each edge, I extract or modify its corresponding value in a global petsc vector. Therefore that vector must
have as many components as edges there are in the network. To extract the value in the vector, I use VecGetArray() and a variable counter that is incremented in each iteration. The array that I obtain in
</span><span style="font-size:13px">VecGetArray() has to be the same size than the edge range. That variable counter starts as 0, so if the array that I obtained in </span><span style="font-size:13px">VecGetArray() is </span><span style="color:rgb(0,0,0)">x_array, </span><span style="color:rgb(0,0,0)">x_array[0]
must be the component in the global vector that corresponds with the start edge given in </span><span style="font-size:13px">DMNetworkGetEdgeRange(</span><span style="font-size:13px">)</span>
<div><br>
</div>
<div>
<div>I need that global petsc vector because I will use it in other operations, it's not just data. Sorry for the confusion. Thanks in advance.</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>This sounds like an assembly operation. The usual paradigm is to compute in the local space, and then communicate to get to the global space. So you would make a PetscSection that had 1 (or some) unknowns on each cell (edge) and then you can use DMCreateGlobal/LocalVector()
and DMLocalToGlobal() to do this.</div>
<div><br>
</div>
<div>Does that make sense?</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div>
<div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div><span style="font-size:13px">Miguel</span></div>
<div><span style="font-size:13px"><br>
</span></div>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Feb 23, 2015 at 9:09 AM, Matthew Knepley <span dir="ltr">
<<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>On Mon, Feb 23, 2015 at 8:42 AM, Miguel Angel Salazar de Troya
<span dir="ltr"><<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<p dir="ltr">Thanks, that will help me. Now what I would like to have is the following: if I have two processors and ten edges, the partitioning results in the first processor having the edges 0-4 and the second processor, the edges 5-9. I also have a global
vector with as many components as edges, 10. How can I partition it so the first processor also has the 0-4 components and the second, the 5-9 components of the vector?</p>
</blockquote>
</span>
<div>I think it would help to know what you want to accomplish. This is how you are proposing to do it.'</div>
<div><br>
</div>
<div>If you just want to put data on edges, DMNetwork has a facility for that already.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div>
<div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<p dir="ltr">Miguel</p>
<div class="gmail_quote">On Feb 23, 2015 8:08 AM, "Abhyankar, Shrirang G." <<a href="mailto:abhyshr@mcs.anl.gov" target="_blank">abhyshr@mcs.anl.gov</a>> wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Helvetica,sans-serif">
<div>Miguel,</div>
<div> One possible way is to store the global numbering of any edge/vertex in the "component" attached to it. Once the mesh gets partitioned, the components are also distributed so you can easily retrieve the global number of any edge/vertex by accessing
its component. This is what is done in the DMNetwork example pf.c although the global numbering is not used for anything. </div>
<div><br>
</div>
<div>Shri</div>
<span>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;border-top-color:rgb(181,196,223)">
<span style="font-weight:bold">From: </span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Mon, 23 Feb 2015 07:54:34 -0600<br>
<span style="font-weight:bold">To: </span>Miguel Angel Salazar de Troya <<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [petsc-users] DMNetworkGetEdgeRange() in parallel<br>
</div>
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Sun, Feb 22, 2015 at 3:59 PM, Miguel Angel Salazar de Troya
<span dir="ltr"><<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Thanks. Once I obtain that Index Set with the routine <span style="font-size:13px">DMPlexCreateCellNumbering() (I assume that the edges in DMNetwork correspond to cells in DMPlex) can I use it to partition a vector with as many components as
edges I have in my network?</span></div>
</blockquote>
<div><br>
</div>
<div>I do not completely understand the question.</div>
<div><br>
</div>
<div>If you want a partition of the edges, you can use DMPlexCreatePartition() and its friend DMPlexDistribute(). What</div>
<div>are you trying to do?</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div><span style="font-size:13px">Thanks</span></div>
<div><span style="font-size:13px">Miguel</span></div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Feb 22, 2015 at 12:15 PM, Matthew Knepley <span dir="ltr">
<<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>On Sun, Feb 22, 2015 at 11:01 AM, Miguel Angel Salazar de Troya
<span dir="ltr"><<a href="mailto:salazardetroya@gmail.com" target="_blank">salazardetroya@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Hi
<div><br>
</div>
<div>I noticed that the routine DMNetworkGetEdgeRange() returns the local indices for the edge range. Is there any way to obtain the global indices? So if my network has 10 edges, the processor 1 has the 0-4 edges and the processor 2, the 5-9 edges, how can
I obtain this information?</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>One of the points of DMPlex is we do not require a global numbering. Everything is numbered</div>
<div>locally, and the PetscSF maps local numbers to local numbers in order to determine ownership.</div>
<div><br>
</div>
<div>If you want to create a global numbering for some reason, you can using DMPlexCreatePointNumbering().</div>
<div>There are also cell and vertex versions that we use for output, so you could do it just for edges as well.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<span>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>Thanks </div>
<span><font color="#888888">
<div>Miguel<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr"><font face="verdana,sans-serif"><b>Miguel Angel Salazar de Troya</b></font><span><font color="#888888"><br>
<font face="arial,helvetica,sans-serif">Graduate Research Assistant<br>
Department of Mechanical Science and Engineering<br>
</font>University of Illinois at Urbana-Champaign<br>
<a href="tel:%28217%29%20550-2360" value="+12175502360" target="_blank">(217) 550-2360</a><br>
<a href="mailto:salaza11@illinois.edu" target="_blank">salaza11@illinois.edu</a></font></span>
<div><br>
</div>
</div>
</div>
</div>
</font></span></div>
</blockquote>
</span></div>
<span><font color="#888888"><br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></font></span></div>
</div>
<span><font color="#888888"></font></span></blockquote>
</div>
<span><font color="#888888"><br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr"><font face="verdana,sans-serif"><b>Miguel Angel Salazar de Troya</b></font><span><font color="#888888"><br>
<font face="arial,helvetica,sans-serif">Graduate Research Assistant<br>
Department of Mechanical Science and Engineering<br>
</font>University of Illinois at Urbana-Champaign<br>
<a href="tel:%28217%29%20550-2360" value="+12175502360" target="_blank">(217) 550-2360</a><br>
<a href="mailto:salaza11@illinois.edu" target="_blank">salaza11@illinois.edu</a></font></span>
<div><br>
</div>
</div>
</div>
</font></span></font></span></div>
<span><font color="#888888"></font></span></blockquote>
<span><font color="#888888"></font></span></div>
<span><font color="#888888"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</blockquote>
</span></div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
<div>
<div><br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</div>
</div>
<span><font color="#888888"></font></span></blockquote>
</div>
<span><font color="#888888"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr"><font face="verdana,sans-serif"><b>Miguel Angel Salazar de Troya</b></font><span><font color="#888888"><br>
<font face="arial,helvetica,sans-serif">Graduate Research Assistant<br>
Department of Mechanical Science and Engineering<br>
</font>University of Illinois at Urbana-Champaign<br>
<a href="tel:%28217%29%20550-2360" value="+12175502360" target="_blank">(217) 550-2360</a><br>
<a href="mailto:salaza11@illinois.edu" target="_blank">salaza11@illinois.edu</a></font></span>
<div><br>
</div>
</div>
</div>
</font></span></div>
</blockquote>
</div>
</div>
</div>
<div>
<div><br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</div>
</div>
<span><font color="#888888"></font></span></blockquote>
</div>
<span><font color="#888888"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr"><font face="verdana,sans-serif"><b>Miguel Angel Salazar de Troya</b></font><span><font color="#888888"><br>
<font face="arial,helvetica,sans-serif">Graduate Research Assistant<br>
Department of Mechanical Science and Engineering<br>
</font>University of Illinois at Urbana-Champaign<br>
<a href="tel:%28217%29%20550-2360" value="+12175502360" target="_blank">(217) 550-2360</a><br>
<a href="mailto:salaza11@illinois.edu" target="_blank">salaza11@illinois.edu</a></font></span>
<div><br>
</div>
</div>
</div>
</font></span></div>
</blockquote>
</div>
</div>
</div>
<div>
<div class="h5"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature">
<div dir="ltr"><font face="verdana,sans-serif"><b>Miguel Angel Salazar de Troya</b></font><span><font color="#888888"><br>
<font face="arial,helvetica,sans-serif">Graduate Research Assistant<br>
Department of Mechanical Science and Engineering<br>
</font>University of Illinois at Urbana-Champaign<br>
(217) 550-2360<br>
<a href="mailto:salaza11@illinois.edu" target="_blank">salaza11@illinois.edu</a></font></span>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
</span>
</body>
</html>