<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Sat, Aug 11, 2018 at 12:02 AM Junchao Zhang <<a href="mailto:jczhang@mcs.anl.gov">jczhang@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">See this example, after multiple casts, the code directly accesses VecScatter_MPI_General's members, with an assumption that from->n is the total number of receive processors. When I separate intranode / internode communications, I have to maintain this assumption.<pre width="80"><span style="color:rgb(0,0,0)"><a name="m_4921736497569505075_line385"></a></span></pre></div></blockquote><div>This is a good example. VecScatter is a flagrant rule violator. It should be rewritten with a proper API</div><div>to obtain this data.</div><div><br></div><div>And if one was going to rewrite it, my personal prejudice would be to try and replace the communication</div><div>part with implementations of PetscSF. Right now SF can handle different data types (Scatter cannot), but</div><div>Scatter has many more modes of communication, such as Alltoall, whereas SF only has p2p and window.</div><div><br></div><div> Matt <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><pre width="80"><span style="color:rgb(0,0,0)"><a name="m_4921736497569505075_line385">385: </a><strong><font color="#4169E1"><a name="m_4921736497569505075_MatMatMultNumeric_MPIDense"></a><a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscErrorCode.html#PetscErrorCode" target="_blank">PetscErrorCode</a> MatMatMultNumeric_MPIDense(<a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/Mat.html#Mat" target="_blank">Mat</a> A,<a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/Mat.html#Mat" target="_blank">Mat</a> B,<a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/Mat.html#Mat" target="_blank">Mat</a> C)</font></strong>
<a name="m_4921736497569505075_line386">386: </a>{</span></pre><pre width="80"><span style="color:rgb(0,0,0)"><a name="m_4921736497569505075_line389">389: </a> Mat_MPIAIJ *aij = (Mat_MPIAIJ*) A->data;
<a name="m_4921736497569505075_line393">393: </a> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatter.html#VecScatter" target="_blank">VecScatter</a> ctx = aij->Mvctx;
<a name="m_4921736497569505075_line394">394: </a> VecScatter_MPI_General *from = (VecScatter_MPI_General*) ctx->fromdata;
<a name="m_4921736497569505075_line395">395: </a> VecScatter_MPI_General *to = (VecScatter_MPI_General*) ctx->todata;
<a name="m_4921736497569505075_line411">411: </a> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscMalloc4.html#PetscMalloc4" target="_blank">PetscMalloc4</a>(B->cmap->N*</span><font color="#ff0000">from->starts[from->n]</font><font color="#000000">,&contents->rvalues,
</font><a name="m_4921736497569505075_line412" style="color:rgb(0,0,0)">412: </a><font color="#000000"> B->cmap->N*</font><font color="#ff0000">to->starts[to->n]</font><font color="#000000">,&contents->svalues,
</font><a name="m_4921736497569505075_line413" style="color:rgb(0,0,0)">413: </a><font color="#000000"> from->n,&contents->rwaits,
</font><a name="m_4921736497569505075_line414" style="color:rgb(0,0,0)">414: </a><font color="#000000"> to->n,&contents->swaits);</font></pre><pre width="80" style="color:rgb(0,0,0)"><br></pre><div><div dir="ltr" class="m_4921736497569505075gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Aug 10, 2018 at 10:33 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Fri, Aug 10, 2018 at 11:29 PM Junchao Zhang <<a href="mailto:jczhang@mcs.anl.gov" target="_blank">jczhang@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It seems we do not have naming conventions for private members.<br clear="all"><div><div dir="ltr" class="m_4921736497569505075m_8301430176723657774m_-5682287504878461405gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"></div></div></div></div></blockquote><div><br></div><div>I am not sure I understand. There are no public members. For private functions, we do have</div><div>a naming convention, but it is newly created, so many of the existing functions break the rules.</div><div><br></div><div> Matt</div><div> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div dir="ltr" class="m_4921736497569505075m_8301430176723657774m_-5682287504878461405gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Aug 10, 2018 at 9:43 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Fri, Aug 10, 2018 at 5:43 PM Junchao Zhang <<a href="mailto:jczhang@mcs.anl.gov" target="_blank">jczhang@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> I met several bugs that remind me to raise this question. In PETSc, object of type A can arbitrarily access object of type B's data. But designer of B may later change the meaning of its data (and of course, update B's interfaces, which are usually local to few files). The designer may think the job is done, but actually it is not. He/she has to grep the code to know where its data members are accessed (that is relatively easy to get) and what is the contract, for example, is an array assumed to be sorted (that is hard to know). With C++, one can use private to minimize data exposure.</div></blockquote><div><br></div><div>This just has to be coding discipline. People should not be accessing private members.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div dir="ltr" class="m_4921736497569505075m_8301430176723657774m_-5682287504878461405m_6671589656297507068m_822683992819083916gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_4921736497569505075m_8301430176723657774m_-5682287504878461405m_6671589656297507068gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_4921736497569505075m_8301430176723657774gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div>