<div dir="ltr">Hi, Philip,<div>   It's my fault.  I should follow up early that this problem was fixed by <a href="https://gitlab.com/petsc/petsc/-/merge_requests/6586">https://gitlab.com/petsc/petsc/-/merge_requests/6586</a>.</div><div>   Could you try petsc/main?</div><div><br></div><div>   Thanks.</div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jun 27, 2023 at 9:30 AM Fackler, Philip <<a href="mailto:facklerpw@ornl.gov">facklerpw@ornl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-4574626591054373336">




<div dir="ltr">
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
Good morning Junchao! I'm following up here to see if there is any update to petsc to resolve this issue, or if we need to come up with a work-around.</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
Thank you,<br>
</div>
<div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_-4574626591054373336Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-size:11pt"><strong>Philip Fackler<br>
</strong></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div><span style="font-size:11pt">Research Software Engineer, Application Engineering Group</span></div>
<div><span style="font-size:11pt">Advanced Computing Systems Research Section</span></div>
<div><span style="font-size:11pt">Computer Science and Mathematics Division<br>
</span></div>
<div><span style="font-size:11pt"><strong>Oak Ridge National Laboratory</strong></span><span style="font-size:11pt"></span><br>
</div>
</div>
</div>
</div>
</div>
<div id="m_-4574626591054373336appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="m_-4574626591054373336divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>><br>
<b>Sent:</b> Wednesday, June 7, 2023 22:45<br>
<b>To:</b> Fackler, Philip <<a href="mailto:facklerpw@ornl.gov" target="_blank">facklerpw@ornl.gov</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>>; Blondel, Sophie <<a href="mailto:sblondel@utk.edu" target="_blank">sblondel@utk.edu</a>>; <a href="mailto:xolotl-psi-development@lists.sourceforge.net" target="_blank">xolotl-psi-development@lists.sourceforge.net</a> <<a href="mailto:xolotl-psi-development@lists.sourceforge.net" target="_blank">xolotl-psi-development@lists.sourceforge.net</a>><br>
<b>Subject:</b> [EXTERNAL] Re: [petsc-users] Initializing kokkos before petsc causes a problem</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi, Philip,
<div>  Thanks for reporting. I will have a look at the issue.<br clear="all">
<div>
<div dir="ltr">
<div dir="ltr">--Junchao Zhang</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Wed, Jun 7, 2023 at 9:30 AM Fackler, Philip via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="ltr">
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
I'm encountering a problem in xolotl. We initialize kokkos before initializing petsc. Therefore...<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
The pointer referenced here:</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_src_vec_is_sf_impls_basic_kokkos_sfkok.kokkos.cxx-23L363&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=i_BmubxwJ4005UQUtS2UbTyKmZwbDAq7aIfA-T4Gnsg&e=" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/vec/is/sf/impls/basic/kokkos/sfkok.kokkos.cxx#L363</a><br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_include_petsc-5Fkokkos.hpp&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=DWQqfHRdrciXObsAIEgF4mGXbMH1xx2vIpcM07ncznU&e=" id="m_-4574626591054373336x_m_8682253095417780210LPNoLPOWALinkPreview" target="_blank"></a></div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
from here:</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_include_petsc-5Fkokkos.hpp&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=DWQqfHRdrciXObsAIEgF4mGXbMH1xx2vIpcM07ncznU&e=" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/include/petsc_kokkos.hpp</a><br>
</div>
<div></div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
remains null because the code to initialize it is skipped here:<br>
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_src_sys_objects_kokkos_kinit.kokkos.cxx-23L28&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=qjM7d6j2BTI04hPQnjucl2H1osvGvortk9kn9e20Hww&e=" id="m_-4574626591054373336x_m_8682253095417780210LPNoLPOWALinkPreview_2" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/kokkos/kinit.kokkos.cxx#L28</a><br>
<div></div>
See line 71.<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
<div></div>
Can this be modified to allow for kokkos to have been initialized by the application before initializing petsc?
<br>
<br>
</div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
Thank you for your help,<br>
</div>
<div>
<div style="font-family:Consolas,Courier,monospace;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_-4574626591054373336x_m_8682253095417780210Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-size:11pt"><strong>Philip Fackler<br>
</strong></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div><span style="font-size:11pt">Research Software Engineer, Application Engineering Group</span></div>
<div><span style="font-size:11pt">Advanced Computing Systems Research Section</span></div>
<div><span style="font-size:11pt">Computer Science and Mathematics Division<br>
</span></div>
<div><span style="font-size:11pt"><strong>Oak Ridge National Laboratory</strong></span><span style="font-size:11pt"></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>

</div></blockquote></div>