<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> It prints Nan to indicate that the time for that event is not known accurately. But the times for the larger events that contain these events are known. So for example the time for KSPSolve is know but not the time for VecNorm. The other numbers in the events, like number of times called etc that are not Nan are correct as displayed.<div class=""><br class=""></div><div class=""> This is done because correctly tracking the times of the individual events requires synchronizations that slow down the entire calculation a bit; for example the time for the KSPSolve will register a longer time then it registers if the smaller events are not timed.</div><div class=""><br class=""></div><div class=""> To display the times of the smaller events use -log_view_gpu_time also but note this will increase the times of the larger events a bit.</div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Oct 5, 2022, at 4:47 PM, Sajid Ali <<a href="mailto:sajidsyed2021@u.northwestern.edu" class="">sajidsyed2021@u.northwestern.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hi PETSc-developers, <div class=""><br class=""></div><div class="">I'm having trouble with getting performance logs from an application that uses PETSc. There are no issues when I run it on a CPU, but every time a GPU is used there is no timing data and almost all times are replaced by times that are just `nan` (on two different clusters). I am attaching the log files for both cases with this email. Could someone explain what is happening here ?<br class=""><br class="">In case it helps, here are the routines used to initialize/finalize the application that also handle initializing/finalizing PETSc and printing the PETSc performance logs to PETSC_VIEWER_STDOUT_WORLD : <a href="https://github.com/fnalacceleratormodeling/synergia2/blob/devel3/src/synergia/utils/utils.h" class="">https://github.com/fnalacceleratormodeling/synergia2/blob/devel3/src/synergia/utils/utils.h</a></div><div class=""><br class=""><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class=""></div>
<div class=""></div>
Thank You,<br class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div style="font-size:12.8px" class="">Sajid Ali (he/him) | Research Associate<br class="">
</div>
<div style="font-size:12.8px" class="">Scientific Computing Division<br class="">
</div>
<div style="font-size:12.8px" class="">Fermi National Accelerator Laboratory<br class="">
</div>
<span style="font-size:12.8px" class=""><a href="http://s-sajid-ali.github.io/" target="_blank" class="">s-sajid-ali.github.io</a></span></div>
</div>
</div>
</div>
</div>
</div></div></div></div></div></div></div></div></div>
<span id="cid:f_l8w31cwk1"><log-gpu></span><span id="cid:f_l8w3oybb1"><log-cpu></span></div></blockquote></div><br class=""></div></body></html>