<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 25, 2020, at 8:36 PM, Jacob Faibussowitsch <<a href="mailto:jacob.fai@gmail.com" class="">jacob.fai@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"> Configure by default should find out the available GPU and build for that sm_* it should not require the user to set this (how the heck is the user going to know what to set?) If I remember correctly there is a utility available that gives this information. </div></blockquote>For CUDA I believe the tool is nvidia-smi. Should make sure this automatic detection works when configuring —with-batch though since login nodes might have different arch than compute.</div></div></blockquote><div><br class=""></div> Would someone buying a machine be that perverse, they put a different GPU on the front end than the back end? Sadly, yes, your point is good, somehow on batch systems one wants the backend GPU information.</div><div><br class=""></div><div> Maybe on Cray systems it is hidden in one of the six million environmental variables they set. </div><div><br class=""></div><div> Barry</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""><div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">Best regards,<br class=""><br class="">Jacob Faibussowitsch<br class="">(Jacob Fai - booss - oh - vitch)<br class="">Cell: (312) 694-3391</div></div>
</div>
<div class=""><br class=""><blockquote type="cite" class=""><div class="">On Sep 25, 2020, at 21:09, Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Configure by default should find out the available GPU and build for that sm_* it should not require the user to set this (how the heck is the user going to know what to set?) If I remember correctly there is a utility available that gives this information. <div class=""><br class=""></div><div class=""> For generic builds like in package distributions I don't know how it should work, ideally all the possibilities would be available in the library and at run time the correct one will be utilized. </div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Sep 25, 2020, at 5:49 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""> '--CUDAFLAGS=-arch=sm_70',<br class=""><div class=""><br class=""></div><div class="">seems to fix this.</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 25, 2020 at 6:31 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">I see kokkos and hyper have a sm_70 flag, but I don't see one for PETSc.<br class=""><br class="">It looks like you have to specify this to get modern atomics to work in Cuda. I get:<div class=""><br class=""></div><div class="">/ccs/home/adams/petsc/include/petscaijdevice.h(99): error: no instance of overloaded function "atomicAdd" matches the argument list<br class=""> argument types are: (double *, double)<br class=""></div><div class=""><br class=""></div><div class="">I tried using a Kokkos configuration, thinking I could get these sm_70 flags, but that did not work.</div><div class=""><br class=""></div><div class="">Any ideas?</div><div class=""><br class=""></div><div class="">Mark</div></div>
</blockquote></div>
</div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></body></html>