<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Nov 28, 2016 at 4:43 AM, Rolf Kuiper <span dir="ltr"><<a href="mailto:kuiper@mpia.de" target="_blank">kuiper@mpia.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Hi Dave,<div><br></div><div>Thanks a lot for your prompt reply! This is even easier than I thought (and that is most likely the reason, why I could not think into this direction) :)</div><div>And yes, I might/should upgrade our PETSc version used (3.1), but it will take me some days to check the full code package.</div><div><br></div><div>Now, I will create my DA (now in 3.6 called DMDA) for the coupled system via</div><div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">DACreate3d(</span>PETSC_COMM_WORLD, <wbr>Periodicity, DA_STENCIL_BOX, <span style="font-variant-ligatures:no-common-ligatures">Nx,</span><span style="font-variant-ligatures:no-common-ligatures"> Ny, Nz, Px, Py, Pz, 4, 1, </span>lx, ly, lz, &MyDA);</div></div><div>instead of</div><div><div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">DACreate3d(</span>PETSC_COMM_WORLD, <wbr>Periodicity, DA_STENCIL_BOX, <span style="font-variant-ligatures:no-common-ligatures">Nx,</span><span style="font-variant-ligatures:no-common-ligatures"> Ny, Nz, Px, Py, Pz, 1, 1, </span>lx, ly, lz, &MyDA);</div></div></div><div>which I have used for the single equation.</div><div><br></div><div>I would like to ask one follow-up question:</div><div>Currently, I loop over my matrix (and vectors) via three for-loops over the 3 spatial directions, and set the columns indices in the following way:</div><div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">for(k){</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"> row.k = k;</div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">0</span><span style="font-variant-ligatures:no-common-ligatures">].k = k;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">].k = k;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">2</span><span style="font-variant-ligatures:no-common-ligatures">].k = k;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">3</span><span style="font-variant-ligatures:no-common-ligatures">].k = k;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">4</span><span style="font-variant-ligatures:no-common-ligatures">].k = k+</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">5</span><span style="font-variant-ligatures:no-common-ligatures">].k = k-</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> for(j){</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"> row.j = j;</div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">0</span><span style="font-variant-ligatures:no-common-ligatures">].j = j;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">].j = j;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">2</span><span style="font-variant-ligatures:no-common-ligatures">].j = j+</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">3</span><span style="font-variant-ligatures:no-common-ligatures">].j = j-</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">4</span><span style="font-variant-ligatures:no-common-ligatures">].j = j;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">5</span><span style="font-variant-ligatures:no-common-ligatures">].j = j;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> for(i){</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"> row.i = i;</div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">0</span><span style="font-variant-ligatures:no-common-ligatures">].i = i+</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">].i = i-</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">1</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">2</span><span style="font-variant-ligatures:no-common-ligatures">].i = i;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">3</span><span style="font-variant-ligatures:no-common-ligatures">].i = i;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">4</span><span style="font-variant-ligatures:no-common-ligatures">].i = i;</span></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> col[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(39,42,216)">5</span><span style="font-variant-ligatures:no-common-ligatures">].i = i;</span></div></div><div>Now for the coupled equations, I should overall additionally loop over the number of DOFs (just 2 in my previous email example). Could you give me an easy example or pseudo-code for the associated assignment of columns (I have the same stencil in each of the submatrices, so for the 2 DOFs in 3D, I would get 7+6=13 column entries per row)? Or can you link me to an existing example within the PETSc help?</div></div></blockquote><div><br></div><div>SNES ex19 uses multiple dofs.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Again, Thanks a lot!</div><div>Rolf</div><div><br></div><div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><br></div></div><div><br></div><div><br><div><blockquote type="cite"><div>Am 24.11.2016 um 22:30 schrieb Dave May <<a href="mailto:dave.mayhem23@gmail.com" target="_blank">dave.mayhem23@gmail.com</a>>:</div><br class="m_9059354790198421053Apple-interchange-newline"><div><div style="white-space:pre-wrap">When you create the DMDA, set the number of DOFs (degrees of freedom) per point to be 2 instead of 1. <br><br>You must be using and ancient version of petsc given the function names you quoted. Consider upgrading to 3.7<br><br>Thanks,<br> Dave</div><br><div class="gmail_quote"><div dir="ltr">On Thu, 24 Nov 2016 at 20:24, Rolf Kuiper <<a href="mailto:kuiper@mpia.de" target="_blank">kuiper@mpia.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="m_9059354790198421053gmail_msg"><div class="m_9059354790198421053gmail_msg">Dear PETSc users,</div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">maybe this is an easy question, but I can’t find the information right away in the user’s guide nor online.</div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">What I am currently doing and which works fine:</div><div class="m_9059354790198421053gmail_msg">To solve a partial differential equation for the quantity q on a parallel distributed grid, which is represented by the Distributed Array MyDA, I am currently creating the associated sparse matrix for the KSP solver via</div><div class="m_9059354790198421053gmail_msg"><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo" class="m_9059354790198421053gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_9059354790198421053gmail_msg">DAGetMatrix(MyDA, MATMPIAIJ, &MyMatrix);</span></div></div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">The solution vector and right hand side vector are created via</div><div class="m_9059354790198421053gmail_msg"><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo" class="m_9059354790198421053gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_9059354790198421053gmail_msg">DACreateGlobalVector(MyDA, &MyRightHandSideVector);</span></div></div><div class="m_9059354790198421053gmail_msg"><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo" class="m_9059354790198421053gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_9059354790198421053gmail_msg">VecDuplicate(<wbr>MyRightHandSideVector, &GlobalSolutionVector);</span></div></div><div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo" class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">The DA is constructed using <span style="font-family:Menlo;font-size:11px;font-variant-ligatures:no-common-ligatures" class="m_9059354790198421053gmail_msg">DACreate3d()</span><span style="font-family:Menlo;font-size:11px" class="m_9059354790198421053gmail_msg"> </span>with the corresponding regular structured grid information. </div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">And here is my problem:</div><div class="m_9059354790198421053gmail_msg">Now, I would like to solve a coupled system of equations for the quantities q1 and q2 on the same grid. I.e., the matrix should just get the double number of rows and columns, the vectors contain twice the number of entries (e.g. first all q1s and then all q2s). And I would like to be sure that the entries of q1 and q2, which are associated with the same grid cell are located on the same processor.</div><div class="m_9059354790198421053gmail_msg">Is there already a pre-defined structures available (such as <span style="font-family:Menlo;font-size:11px" class="m_9059354790198421053gmail_msg">MATMPIAIJ</span>) within PETSc to enlarge such a single equation to store the entries of coupled equations? Such as</div><div class="m_9059354790198421053gmail_msg"><span style="font-family:Menlo;font-size:11px" class="m_9059354790198421053gmail_msg">DACreateTwiceTheGlobalVector()</span><wbr>?</div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">The equation is (simplified) of the form</div><div class="m_9059354790198421053gmail_msg">d/dt q1 + grad q2 = 0</div><div class="m_9059354790198421053gmail_msg">d/dt q2 + f(q1) = 0</div><div class="m_9059354790198421053gmail_msg">with an arbitrary function f depending on q1.</div><div class="m_9059354790198421053gmail_msg"><br class="m_9059354790198421053gmail_msg"></div><div class="m_9059354790198421053gmail_msg">Thanks a lot for your help in advance,</div><div class="m_9059354790198421053gmail_msg">Rolf</div></div></blockquote></div>
</div></blockquote></div><br></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>