<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Can valgrind run and help with OpenMP?<div class=""><br class=""></div><div class=""> You can run in the debugger and find any calls to the options checking inside your code block and comment them all out to see if that eliminates the problem.</div><div class=""><br class=""></div><div class=""> Also generically how safe is CUDA inside OpenMP? That is with multiple threads calling CUDA stuff?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""></div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jan 18, 2021, at 7:04 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">Added this w/o luck:</div><div class=""><br class=""></div><div class="">#if defined(PETSC_HAVE_CUDA)<br class=""> ierr = PetscOptionsCheckCUDA(logView);CHKERRQ(ierr);<br class="">#if defined(PETSC_HAVE_THREADSAFETY)<br class=""> ierr = PetscCUPMInitializeCheck();CHKERRQ(ierr);<br class="">#endif<br class="">#endif<br class=""></div><div class=""> </div><div class="">Do you think I should keep this in or take it out? Seems like a good idea and when it all works we can see if we can make it lazy.</div><div class=""><br class=""></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
1) Calling PetscOptions inside threads. I looked quickly at the code and it seems like it should be ok but perhaps not. This is one reason why having stuff like PetscOptionsBegin inside a low-level creation VecCreate_SeqCUDA_Private is normally not done in PETSc. Eventually this needs to be moved or reworked. <br class="">
<br class=""></blockquote><div class=""><br class=""></div><div class="">I will try this next. It is hard to see the stack here. I think I will put it in ddt and put a breakpoint PetscOptionsEnd_Private. Other ideas welcome.</div><div class=""><br class=""></div><div class="">Mark</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
2) PetscCUDAInitializeCheck is not thread safe.If it is being call for the first timeby multiple threads there can be trouble. So edit init.c and under<br class="">
<br class="">
#if defined(PETSC_HAVE_CUDA)<br class="">
ierr = PetscOptionsCheckCUDA(logView);CHKERRQ(ierr);<br class="">
#endif<br class="">
<br class="">
#if defined(PETSC_HAVE_HIP)<br class="">
ierr = PetscOptionsCheckHIP(logView);CHKERRQ(ierr);<br class="">
#endif<br class="">
<br class="">
put in <br class="">
#if defined thread safety<br class="">
PetscCUPMInitializeCheck<br class="">
#endif<br class="">
<br class=""></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
this will force the initialize to be done before any threads are used <br class=""></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br class=""></blockquote></div></div>
</div></blockquote></div><br class=""></div></body></html>