<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Jan 26, 2017 at 10:20 AM, Xiangdong <span dir="ltr"><<a href="mailto:epscodes@gmail.com" target="_blank">epscodes@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thanks, Matt. For a cell in DMDA 3d with global id (ix,iy,iz), what is the global row id of that cell corresponding to the matrix generated by DMCreateMatrix? It is not always ix + iy*Nx + iz*Ny*Nx for different da_processors_xyz, is it? How can I obtain that global row id?</div></div></blockquote><div><br></div><div>It is not simple:</div><div><br></div><div> 1) Figure out the process division using</div><div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-master/docs/manualpages/DMDA/DMDAGetInfo.html">http://www.mcs.anl.gov/petsc/petsc-master/docs/manualpages/DMDA/DMDAGetInfo.html</a></div><div><br></div><div> 2) Figure out which process p your i,j,k in on</div><div><br></div><div> You have to march through the processes, knowing how big the chunk of indices is on each,</div><div> which you get from 1) and</div><div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-master/docs/manualpages/DMDA/DMDAGetOwnershipRanges.html">http://www.mcs.anl.gov/petsc/petsc-master/docs/manualpages/DMDA/DMDAGetOwnershipRanges.html</a></div><div><br></div><div> 3) Add up the sizes from all process less than p, and then the offset on process p. This the global offset</div><div><br></div><div>This is not a scalable thing to do, which is why we do not include it in the API.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thanks.</div><div><br></div><div>Xiangdong</div><div><div><br></div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 25, 2017 at 10:53 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Wed, Jan 25, 2017 at 9:17 AM, Xiangdong <span dir="ltr"><<a href="mailto:epscodes@gmail.com" target="_blank">epscodes@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello everyone,<div><br></div><div>I have a question on setting matrix entries which are not in the stencil width. Take ksp ex45.c as an example, </div><div><a href="http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex45.c.html" target="_blank">http://www.mcs.anl.gov/petsc/p<wbr>etsc-current/src/ksp/ksp/examp<wbr>les/tutorials/ex45.c.html</a></div><div><br></div><div>Instead of setting up the standard 7-point stencil, now for each cell, the matrix also has a additional dependency on the cell (Mx, My, Mz). Namely, for each row, the col corresponding to (Mx, My, Mz) is always nonzero. I modify the example code to add this entries:</div><div><br></div><div><div>+ MatSetOption(B,MAT_NEW_NONZER<wbr>O_LOCATIONS,PETSC_TRUE);</div><div>+ MatSetOption(jac,MAT_NEW_NONZ<wbr>ERO_LOCATIONS,PETSC_TRUE);</div></div><div><br></div><div><div>+<span class="gmail-m_-4329815367029045524m_-3456302684694579183m_-8867078549042271816gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> v[7] = 100; col[7].i = mx-1; col[7].j = my-1; col[7].k = mz-1;</div><div>+<span class="gmail-m_-4329815367029045524m_-3456302684694579183m_-8867078549042271816gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> ierr = MatSetValuesStencil(B,1,&row,8<wbr>,col,v,INSERT_VALUES);CHKERRQ(<wbr>ierr);</div></div><div><br></div><div>It is okay to for np=1, but crash for np>=2 with the error message:</div></div></blockquote><div><br></div></span><div>You can do this, but</div><div><br></div><div>1) You cannot use MatSetStencil, since your entry is not actually in your stencil. You will have to make a separate call to MatSetValues() using the global index.</div><div><br></div><div>2) The nonzero pattern we have allocated will be wrong. You will have to set the MatOption which gives an error on new nonzeros to PETSC_FALSE.</div><div><br></div><div>3) You will have a dense row in your Jacobian, which is hard to make perform well, and also stymies most preconditioners.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--</div><div>[0]PETSC ERROR: Argument out of range</div><div>[0]PETSC ERROR: Local index 342 too large 244 (max) at 7</div></div><div><br></div><div><div>[0]PETSC ERROR: #1 ISLocalToGlobalMappingApply() line 423 in petsc-3.7.4/src/vec/is/utils/i<wbr>sltog.c</div><div>[0]PETSC ERROR: #2 MatSetValuesLocal() line 2052 in petsc-3.7.4/src/mat/interface/<wbr>matrix.c</div><div>[0]PETSC ERROR: #3 MatSetValuesStencil() line 1447 in petsc-3.7.4/src/mat/interface/<wbr>matrix.c</div><div>[0]PETSC ERROR: #4 ComputeMatrix() line 151 in extest.c</div></div><div><br></div><div>Can I add new entries to the cols not in the stencil width into the dmda matrix or Jacobian?</div><div><br></div><div>Attached please find the modifed ex45 example, the diff file as well as the run log.</div><div><br></div><div>Thanks for your help.</div><div><br></div><div>Best,</div><div>Xiangdong</div><div><br></div></div>
</blockquote></span></div><span class="gmail-m_-4329815367029045524HOEnZb"><font color="#888888"><br><br clear="all"><span class="gmail-HOEnZb"><font color="#888888"><div><br></div>-- <br><div class="gmail-m_-4329815367029045524m_-3456302684694579183gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></font></span></div></div>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>