<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Consolas, Courier, monospace; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Good morning Junchao! I'm following up here to see if there is any update to petsc to resolve this issue, or if we need to come up with a work-around.</div>
<div style="font-family: Consolas, Courier, monospace; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Consolas, Courier, monospace; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Thank you,<br>
</div>
<div class="elementToProof">
<div style="font-family: Consolas, Courier, monospace; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-size:11pt"><strong>Philip Fackler<br>
</strong></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div><span style="font-size:11pt">Research Software Engineer, Application Engineering Group</span></div>
<div><span style="font-size:11pt">Advanced Computing Systems Research Section</span></div>
<div><span style="font-size:11pt">Computer Science and Mathematics Division<br>
</span></div>
<div><span style="font-size:11pt"><strong>Oak Ridge National Laboratory</strong></span><span style="font-size:11pt"></span><br>
</div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Junchao Zhang <junchao.zhang@gmail.com><br>
<b>Sent:</b> Wednesday, June 7, 2023 22:45<br>
<b>To:</b> Fackler, Philip <facklerpw@ornl.gov><br>
<b>Cc:</b> petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov>; Blondel, Sophie <sblondel@utk.edu>; xolotl-psi-development@lists.sourceforge.net <xolotl-psi-development@lists.sourceforge.net><br>
<b>Subject:</b> [EXTERNAL] Re: [petsc-users] Initializing kokkos before petsc causes a problem</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi, Philip,
<div> Thanks for reporting. I will have a look at the issue.<br clear="all">
<div>
<div dir="ltr" class="x_gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">--Junchao Zhang</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Wed, Jun 7, 2023 at 9:30 AM Fackler, Philip via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div class="x_msg8682253095417780210">
<div dir="ltr">
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
I'm encountering a problem in xolotl. We initialize kokkos before initializing petsc. Therefore...<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
The pointer referenced here:</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_src_vec_is_sf_impls_basic_kokkos_sfkok.kokkos.cxx-23L363&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=i_BmubxwJ4005UQUtS2UbTyKmZwbDAq7aIfA-T4Gnsg&e=" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/vec/is/sf/impls/basic/kokkos/sfkok.kokkos.cxx#L363</a><br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_include_petsc-5Fkokkos.hpp&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=DWQqfHRdrciXObsAIEgF4mGXbMH1xx2vIpcM07ncznU&e=" id="x_m_8682253095417780210LPNoLPOWALinkPreview" target="_blank"></a></div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
from here:</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_include_petsc-5Fkokkos.hpp&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=DWQqfHRdrciXObsAIEgF4mGXbMH1xx2vIpcM07ncznU&e=" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/include/petsc_kokkos.hpp</a><br>
</div>
<div></div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
remains null because the code to initialize it is skipped here:<br>
<a href="https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_blob_main_src_sys_objects_kokkos_kinit.kokkos.cxx-23L28&d=DwMFaQ&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=mm5WnvW66V9aFezISyisTVEya_MgbI4YjpAuorp4BHdklRfAD8kTzaDBY3VlzI-4&s=qjM7d6j2BTI04hPQnjucl2H1osvGvortk9kn9e20Hww&e=" id="x_m_8682253095417780210LPNoLPOWALinkPreview_2" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/kokkos/kinit.kokkos.cxx#L28</a><br>
<div></div>
See line 71.<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<br>
<div></div>
Can this be modified to allow for kokkos to have been initialized by the application before initializing petsc?
<br>
<br>
</div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
Thank you for your help,<br>
</div>
<div>
<div style="font-family:Consolas,Courier,monospace; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_m_8682253095417780210Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-size:11pt"><strong>Philip Fackler<br>
</strong></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div><span style="font-size:11pt">Research Software Engineer, Application Engineering Group</span></div>
<div><span style="font-size:11pt">Advanced Computing Systems Research Section</span></div>
<div><span style="font-size:11pt">Computer Science and Mathematics Division<br>
</span></div>
<div><span style="font-size:11pt"><strong>Oak Ridge National Laboratory</strong></span><span style="font-size:11pt"></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>