<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jun 20, 2020 at 12:24 PM Barry Smith <<a href="mailto:bsmith@petsc.dev">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;"><div><br></div> Junchao,<div><br></div><div> This is a good bug fix. It solves the problem when PETSc initialize is called many times. </div><div><br></div><div> There is another fix you can do to limit PETSc mpiuni running out of attributes inside a single PETSc run:</div><div><br></div><div><br></div><div><div>int MPI_Comm_create_keyval(MPI_Copy_function *copy_fn,MPI_Delete_function *delete_fn,int *keyval,void *extra_state)</div><div>{</div><div> </div><div> if (num_attr >= MAX_ATTR){</div><div> for (i=0; i<num_attr; i++) {</div><div> if (!attr_keyval[i].extra_state) {</div></div></div></blockquote><div>attr_keyval[i].extra_state is provided by user (could be NULL). We can not rely on it. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;"><div><div> /* reuse this slot */</div><div> attr_keyval[i].extra_state = extra_state;</div><div> attr_keyval[i.]del = delete_fn;</div><div> *keyval = i;</div><div> return MPI_SUCCESS;</div><div> }</div><div> }</div><div> return MPIUni_Abort(MPI_COMM_WORLD,1);</div><div>}</div><div> return MPIUni_Abort(MPI_COMM_WORLD,1);</div><div> attr_keyval[num_attr].extra_state = extra_state;</div><div> attr_keyval[num_attr].del = delete_fn;</div><div> *keyval = num_attr++;</div><div> return MPI_SUCCESS;</div><div>}</div></div><div><br></div><div> This will work if the user creates tons of attributes but is constantly deleting some as they new ones. So long as the number outstanding at one time is < MAX_ATTR)</div><div><br></div><div>Barry</div><div><br></div><div><br></div><div><br></div><div><br><div><br><blockquote type="cite"><div>On Jun 20, 2020, at 10:54 AM, Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:</div><br><div><div dir="ltr">I don't understand what your session means. Let's try this patch<div><br></div><div>diff --git a/src/sys/mpiuni/mpi.c b/src/sys/mpiuni/mpi.c<br>index d559a513..c058265d 100644<br>--- a/src/sys/mpiuni/mpi.c<br>+++ b/src/sys/mpiuni/mpi.c<br>@@ -283,6 +283,7 @@ int MPI_Finalize(void)<br> MPI_Comm_free(&comm);<br> comm = MPI_COMM_SELF;<br> MPI_Comm_free(&comm);<br>+ num_attr = 1; /* reset the counter */<br> MPI_was_finalized = 1;<br> return MPI_SUCCESS;<br> }<br><div><br></div><div><br clear="all"><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jun 20, 2020 at 10:48 AM Sam Guo <<a href="mailto:sam.guo@cd-adapco.com" target="_blank">sam.guo@cd-adapco.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Typo: I mean “Assuming initializer is only needed once for entire session”<br><br>On Saturday, June 20, 2020, Sam Guo <<a href="mailto:sam.guo@cd-adapco.com" target="_blank">sam.guo@cd-adapco.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Assuming finalizer is only needed once for entire session(?), I can put initializer into the static block to call it once but where do I call finalizer?<div><br></div><div><br>On Saturday, June 20, 2020, Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The counter num_attr should be recycled. But first try to call PETSc initialize/Finalize only once to see it fixes the error.<br clear="all"><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jun 20, 2020 at 12:48 AM Sam Guo <<a href="mailto:sam.guo@cd-adapco.com" target="_blank">sam.guo@cd-adapco.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">To clarify, I call PETSc initialize and PETSc finalize everytime I call SLEPc:<div><br><div><pre width="80" style="white-space:pre-wrap"> PetscInitializeNoPointers(argc,args,nullptr,nullptr);<br></pre><pre width="80" style="white-space:pre-wrap"> SlepcInitialize(&argc,&args,static_cast<char*>(nullptr),help);</pre><pre width="80" style="white-space:pre-wrap"> //calling slepc</pre><pre width="80" style="white-space:pre-wrap"> SlepcFinalize();</pre><pre width="80" style="white-space:pre-wrap"><span style="font-family:Arial,Helvetica,sans-serif;color:rgb(34,34,34)"> PetscFinalize();</span> </pre></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jun 19, 2020 at 10:32 PM Sam Guo <<a href="mailto:sam.guo@cd-adapco.com" target="_blank">sam.guo@cd-adapco.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear PETSc team,<div> When I called SLEPc multiple time, I eventually got following error: </div><div><br></div><div><span style="color:rgb(80,0,80)">MPI operation not supported by PETSc's sequential MPI wrappers<br></span>[0]PETSC ERROR: #1 PetscInitialize() line 967 in ../../../petsc/src/sys/objects/pinit.c<br>[0]PETSC ERROR: #2 SlepcInitialize() line 262 in ../../../slepc/src/sys/slepcinit.c<br>[0]PETSC ERROR: #3 SlepcInitializeNoPointers() line 359 in ../../../slepc/src/sys/slepcinit.c<span style="color:rgb(80,0,80)"><br>PETSC ERROR: Logging has not been enabled.<br>You might have forgotten to call PetscInitialize().</span> <br></div><div><br></div><div> I debugged: it is because of following in petsc/src/sys/mpiuni/mpi.c</div><div><br></div><div><div>if (num_attr >= MAX_ATTR)<br></div></div><div><br></div><div>in function int MPI_Comm_create_keyval(MPI_Copy_function *copy_fn,MPI_Delete_function *delete_fn,int *keyval,void *extra_state)</div><div><br></div><div>num_attr is declared static and keeps increasing every time MPI_Comm_create_keyval is called.</div><div><br></div><div>I am using petsc 3.11.3 but found 3.13.2 has the same logic.</div><div><br></div><div>Is this a bug or I didn't use it correctly?</div><div><br></div><div>Thanks,</div><div>Sam</div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote>
</blockquote></div>
</div></blockquote></div><br></div></div></blockquote></div></div>