<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Hi Matthew</div>
<div><br>
</div>
<div>Thanks for your answer and your fix. It works :)))</div>
<div><br>
</div>
<div>Kind regards,</div>
<div>Morten</div>
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">Fra: </span>Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>><br>
<span style="font-weight:bold">Dato: </span>Thursday 14 July 2016 at 00:03<br>
<span style="font-weight:bold">Til: </span>Morten Nobel-Joergensen <<a href="mailto:mono@dtu.dk">mono@dtu.dk</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>><br>
<span style="font-weight:bold">Emne: </span>Re: [petsc-users] Distribution of DMPlex for FEM<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Jul 13, 2016 at 3:57 AM, Morten Nobel-Jørgensen <span dir="ltr">
<<a href="mailto:mono@dtu.dk" target="_blank">mono@dtu.dk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div style="word-wrap:break-word">
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">I’m having problems distributing a simple FEM model using DMPlex. For test case I use 1x1x2 hex box elements (/cells) with 12 vertices. Each vertex has one DOF.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">When I distribute the system to two processors, each get a single element and the local vector has the size 8 (one DOF for each vertex of a hex box) as expected.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">My problem is that when I manually assemble the global stiffness matrix (a 12x12 matrix) it seems like my ghost values are ignored. I’m sure that I’m missing something obvious but cannot
see what it is.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">In the attached example, I’m assembling the global stiffness matrix using a simple local stiffness matrix of ones. This makes it very easy to see if the matrix is assembled correctly.
If I run it on one process, then global stiffness matrix consists of 0’s, 1’s and 2’s and its trace is 16.0. But if I run it distributed on on two, then it consists only of 0's and 1’s and its trace is 12.0.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">I hope that somebody can spot my mistake and help me in the right direction :)</div>
</div>
</blockquote>
<div><br>
</div>
<div>This is my fault, and Stefano Zampini had already tried to tell me this was broken. I normally use DMPlexMatSetClosure(), which handles global indices correctly.</div>
<div>I have fixed this in the branch</div>
<div><br>
</div>
<div> knepley/fix-plex-l2g</div>
<div><br>
</div>
<div>which is also merged to 'next'. I am attaching a version of your sample where all objects are freed correctly. Let me know if that works for you.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div style="word-wrap:break-word">
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">Kind regards,</div>
<div style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:14px">Morten</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>