<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Apr 25, 2021, at 10:04 PM, 王 杰 <<a href="mailto:wangjieneu@hotmail.com" class="">wangjieneu@hotmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">Thanks very much for your reply!</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">1、</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">In the first process, the global index starts from 0, and the local index also starts from 0. The MatCreateAij and MatSetValues work well.</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">However, in the second process, the minimum global index is<span class="Apple-converted-space"> </span><span style="color: rgb(237, 92, 87);" class=""><b class="">601</b></span>, which doesn't coincide with the local index that starts from 0. Should I give a global_offset '<span style="color: rgb(237, 92, 87);" class=""><b class="">601</b></span>' when prepare the parameters<span class="Apple-converted-space"> </span><span style="color: rgb(237, 92, 87);" class=""><b class="">*d_nnz</b></span><span class="Apple-converted-space"> </span>and<span class="Apple-converted-space"> </span><span style="color: rgb(237, 92, 87);" class=""><b class="">*o_nnz</b></span>?</div></div></blockquote><div><br class=""></div> No, <span class="" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(237, 92, 87);"><b class="">d_nnz</b></span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class=""> </span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">and</span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class=""> </span><span class="" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(237, 92, 87);"><b class="">*o_nnz don't have anything to do with global indices (that is determined completely by local indices set on each ran).</b></span></div><div><font color="#ed5c57" face="Calibri, Helvetica, sans-serif" size="3" class=""><span style="caret-color: rgb(237, 92, 87);" class=""><b class=""><br class=""></b></span></font><blockquote type="cite" class=""><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">2、</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">In the first process，m and n are 1600 but in the second process they are 900. Is it ok to give different m and n in the two process which related to the same matrix d_A ? For example, <span style="background-color: rgb(255, 255, 255); display: inline !important;" class="">(1600,1600)for the first process and (900,900) for the second process?</span></div></div></blockquote><div><br class=""></div> Yes</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">3、</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">The global index in both process are not continuous. Is it necessory to use Application Orderings routines to make it continuous? </div></div></blockquote><div><br class=""></div> PETSc global indices for matrices and vectors are always continuous; how you choose to manage your numbering of your own internal objects may or may not relate to the ordering PETSc users. When you use MatSetValue() and VecSetValues() you must use PETSc's continuous ordering. You can set MatSetLocalToGlobalMapping and VecSetLocalToGlobalMapping to allow you to use MatSetValuesLocal and VecSetValuesLocal using arbitrary local orderings.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">4、</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">Is it necessory to difine the ghost nodes explicitily with the rountine <span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class=""><u class="">DMDACreate2d</u>(MPLComm comm, DMBoundaryType bx, DMBoundaryType by, DMDAStencilType stype, Petscint M, Petscint N, Petscint m, Petscint n, Petscint do£, Petscint s, canst Petscint lx[], canst Petscint ly[],<span class="Apple-converted-space"> </span></span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">DM '''da)?</span></div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class=""><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">Or I just to assign the nodes with the same global indices to both process, and the Petsc will generate ghost nodes automatically?</span></div></div></blockquote><div><br class=""></div> If you use DMDACreate2d() which only makes sense for 2d structure grids the arguments determine the exact ghosting, basically the stencil width s and type of boundary conditions determine the ghosting. You cannot use another ghosting. But global indexing has nothing to do with ghosting and any process can use any global indices it would like and the Mat/Vec handle the needed communication.</div><div><br class=""></div><div> Barry</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div id="appendonsend" class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;" class=""><br class=""></div><hr tabindex="-1" style="display: inline-block; width: 1180.890625px;" class=""><div id="divRplyFwdMsg" dir="ltr" class=""><font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b class="">发件人:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>><br class=""><b class="">发送时间:</b><span class="Apple-converted-space"> </span>2021年4月25日 17:22<br class=""><b class="">收件人:</b><span class="Apple-converted-space"> </span>王 杰 <<a href="mailto:wangjieneu@hotmail.com" class="">wangjieneu@hotmail.com</a>><br class=""><b class="">抄送:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>><br class=""><b class="">主题:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Can I assign different local number of row and column to the parallel matrix in different process?</font><div class=""> </div></div><div class="" style="word-wrap: break-word; line-break: after-white-space;"><div class=""><br class=""></div> Yes; but note that for square matrices using n == m almost always makes sense, otherwise the input vector to the matrix-vector product will have a different layout than the output vector. Code code is difficult to manage. <br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Apr 23, 2021, at 9:29 PM, 王 杰 <<a href="mailto:wangjieneu@hotmail.com" class="">wangjieneu@hotmail.com</a>> wrote:</div><br class="x_Apple-interchange-newline"><div class=""><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">hello,</div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><br class=""></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">I use petsc 3.14.5 with OpenMPI 3.1.6 in Ubuntu 18.04.5.</div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><br class=""></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">I don't well understand the routine <span class="x_Apple-converted-space"> </span><u class="" style="text-decoration-line: underline; background-color: rgb(255, 255, 255);">MatCreateAIJ</u><span class="" style="text-decoration-line: underline; background-color: rgb(255, 255, 255); display: inline !important;">(</span><span class="" style="background-color: rgb(255, 255, 255); display: inline !important;">MPI_Comm comm,</span><span class="" style="background-color: rgb(255, 255, 255); font-family: calibri, helvetica, sans-serif; font-size: 12pt; color: rgb(200, 38, 19); display: inline !important;"><b class="">PetscInt</b></span><span class="" style="background-color: rgb(255, 255, 255); color: rgb(200, 38, 19); display: inline !important;"><b class=""><span class="x_Apple-converted-space"> </span>m</b></span><span class="" style="background-color: rgb(255, 255, 255); display: inline !important;">, </span><span class="" style="background-color: rgb(255, 255, 255); font-family: calibri, helvetica, sans-serif; font-size: 12pt; color: rgb(200, 38, 19); display: inline !important;"><b class="">PetscInt<span class="x_Apple-converted-space"> </span></b></span><span class="" style="background-color: rgb(255, 255, 255); font-family: calibri, helvetica, sans-serif; font-size: 12pt; color: rgb(200, 38, 19); display: inline !important;"><b class="">n,</b></span><span class="" style="background-color: rgb(255, 255, 255); display: inline !important;">PetscInt M,PetscInt N,PetscInt d_nz,PetscInt *d_nnz, PetscInt o_nz,PetscInt *o_nnz,Mat *A</span><span class="" style="text-decoration-line: underline; background-color: rgb(255, 255, 255); display: inline !important;">)</span>. </div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><br class=""></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">I have tow process. The total number of global index is 2188. The first one deal with the global index 0~1400 with ghost node 1401~1500 and 2001~2188. The second process deal with globla index 1400~2100 with ghost node 601~700 and 701~800. This is dynamiclly assigned by load balancer.</div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"> </div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">I want to assign the first 1400 rows to the first process and the remained 788 rows in the second process. Can I give different values to m and n in the routine <u class="" style="background-color: rgb(255, 255, 255);">MatCreateAIJ</u><span class="" style="background-color: rgb(255, 255, 255);">in different process?</span></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><span class="" style="background-color: rgb(255, 255, 255);"><br class=""></span></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><span class="" style="background-color: rgb(255, 255, 255);">Thanks.</span></div><div class="" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;"><span class="" style="background-color: rgb(255, 255, 255);"> Wang Jie</span></div></div></blockquote></div></div></div></div></blockquote></div><br class=""></body></html>