<div dir="ltr"><div><br></div><div>From the code, it seems you need to let your users set env var <span style="color:rgb(163,21,21);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre">PETSC_LIB_DIR </span>to the library location, or<span style="color:rgb(163,21,21);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre"> PETSC_DIR </span>to the prefix path and <span style="color:rgb(163,21,21);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre">PETSC_ARCH </span>to ""</div><div><br></div><div><span style="font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre;color:rgb(0,0,255)">static</span><span style="color:rgb(0,0,0);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre"> PetscErrorCode PetscLoadDynamicLibrary(</span><span style="font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre;color:rgb(0,0,255)">const</span><span style="color:rgb(0,0,0);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre"> </span><span style="font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre;color:rgb(0,0,255)">char</span><span style="color:rgb(0,0,0);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;white-space:pre"> *name, PetscBool *found)</span><br></div><div><div style="color:rgb(0,0,0);font-family:Menlo,Monaco,"Courier New",monospace;font-size:14px;line-height:21px;white-space:pre"><div>{</div><div> <span style="color:rgb(0,0,255)">char</span> libs[<span style="color:rgb(0,0,255)">PETSC_MAX_PATH_LEN</span>], dlib[<span style="color:rgb(0,0,255)">PETSC_MAX_PATH_LEN</span>];</div><br><div> <span style="color:rgb(0,0,255)">PetscFunctionBegin</span>;</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscStrncpy(libs, <span style="color:rgb(163,21,21)">"${PETSC_LIB_DIR}/libpetsc"</span>, <span style="color:rgb(0,0,255)">sizeof</span>(libs)));</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscStrlcat(libs, name, <span style="color:rgb(0,0,255)">sizeof</span>(libs)));</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscDLLibraryRetrieve(PETSC_COMM_WORLD, libs, dlib, <span style="color:rgb(9,134,88)">1024</span>, found));</div><div> <span style="color:rgb(0,0,255)">if</span> (*found) {</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscDLLibraryAppend(PETSC_COMM_WORLD, &PetscDLLibrariesLoaded, dlib));</div><div> } <span style="color:rgb(0,0,255)">else</span> {</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscStrncpy(libs, <span style="color:rgb(163,21,21)">"${PETSC_DIR}/${PETSC_ARCH}/lib/libpetsc"</span>, <span style="color:rgb(0,0,255)">sizeof</span>(libs)));</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscStrlcat(libs, name, <span style="color:rgb(0,0,255)">sizeof</span>(libs)));</div><div> <span style="color:rgb(0,0,255)">PetscCall</span>(PetscDLLibraryRetrieve(PETSC_COMM_WORLD, libs, dlib, <span style="color:rgb(9,134,88)">1024</span>, found));</div><div> <span style="color:rgb(0,0,255)">if</span> (*found) <span style="color:rgb(0,0,255)">PetscCall</span>(PetscDLLibraryAppend(PETSC_COMM_WORLD, &PetscDLLibrariesLoaded, dlib));</div><div> }</div><div> <span style="color:rgb(0,0,255)">PetscFunctionReturn</span>(<span style="color:rgb(0,0,255)">PETSC_SUCCESS</span>);</div><div>}</div></div></div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 21, 2023 at 4:59 PM Di Miao via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg4225664414938108325">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="m_4225664414938108325WordSection1">
<p class="MsoNormal">Hi Barry,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I found the following snippet in PetscInitialize_DynamicLibraries:<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">#if defined(PETSC_HAVE_THREADSAFETY)<u></u><u></u></p>
<p class="MsoNormal"> /* These must be all initialized here because it is not safe for individual threads to call these initialize routines */<u></u><u></u></p>
<p class="MsoNormal"> preload = PETSC_TRUE;<u></u><u></u></p>
<p class="MsoNormal">#endif<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">and ‘PetscLoadDynamicLibrary("", &found)’ is called only if preload==true and petsc is built as a shared lib. We have not faced this issue before we add ‘—with-thread-safety’.
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">What we are doing is:<u></u><u></u></p>
<p class="MsoNormal">Compile petsc/slepc as shared libraries, ship both executable and libpetsc.so libslepc.so to other users.
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Does this error message indicate that what we are doing is not robust and it is better to use static libraries instead?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thank you<u></u><u></u></p>
<p class="MsoNormal">Di<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> <br>
<b>Sent:</b> Thursday, September 21, 2023 2:44 PM<br>
<b>To:</b> Di Miao <<a href="mailto:dimiao@synopsys.com" target="_blank">dimiao@synopsys.com</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>; Udit Pillai <<a href="mailto:uditp@synopsys.com" target="_blank">uditp@synopsys.com</a>><br>
<b>Subject:</b> Re: [petsc-users] 'Unable to locate PETSc dynamic library' error<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"> The recommended approach is to use --with-prefix=directory where you want PETSc installed instead of copying files manually after configure and make<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> I don't see why —with-thread-safety would affect this but<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">On Sep 21, 2023, at 5:07 PM, Di Miao via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">To whom it may concern,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">In our build system, PETSc/SLEPc is compiled in one place and the *.so lib are then moved to another place for future use. Before we add the ‘—with-thread-safety’ flag, this works well. Now, we would like to run PETSc on multiple threads.
As suggested by the manual, we added ‘—with-thread-safety’ flag. The udpated libpetsc.so leads to the following error message:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Unable to locate PETSc dynamic library<span class="m_4225664414938108325apple-converted-space"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> You cannot move the dynamic libraries!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">My question is other than compile PETSc as a static library, is there any other solutions that can let us compile libpetsc.so in one place and distribute it to other folders without causing this error?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thank you,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Di<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Unable to open file<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Unable to locate PETSc dynamic library<span class="m_4225664414938108325apple-converted-space"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> You cannot move the dynamic libraries!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: See<span class="m_4225664414938108325apple-converted-space"> </span><a href="https://urldefense.com/v3/__https:/petsc.org/release/faq/__;!!A4F2R9G_pg!fam7m9EAcUH3ksmrAcDKqw2b56d-5BMr-p7YSieNL7fFnEN9xuYzPbkyxhVcvvp16WSAXnrY-j8Q5wI00Q$" target="_blank">https://petsc.org/release/faq/</a><span class="m_4225664414938108325apple-converted-space"> </span>for
trouble shooting.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Petsc Release Version 3.19.2, Jun 01, 2023<span class="m_4225664414938108325apple-converted-space"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Unknown Name on a arch-linux-c-opt named<span class="m_4225664414938108325apple-converted-space"> </span><a href="http://tcadprod39.synopsys.com/" target="_blank">tcadprod39.synopsys.com</a><span class="m_4225664414938108325apple-converted-space"> </span>by tcad Wed Sep 20 02:37:04
2023<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Configure options <a href="mailto:--with-mpi-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../tcad_mpi_mpich" target="_blank">
--with-mpi-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../tcad_mpi_mpich</a> --with-64-bit-indices=true
<a href="mailto:--with-blaslapack-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../mkl" target="_blank">
--with-blaslapack-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../mkl</a> --with-shared-libraries --with-clean=true --with-debugging=0
<a href="mailto:--with-mkl_pardiso-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../mkl" target="_blank">
--with-mkl_pardiso-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../mkl</a> COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3
<a href="mailto:--download-mumps=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/MUMPS_5.6.0.tar.gz" target="_blank">
--download-mumps=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/MUMPS_5.6.0.tar.gz</a>
<a href="mailto:--download-scalapack=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/v2.2.0.tar.gz" target="_blank">
--download-scalapack=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/v2.2.0.tar.gz</a>
<a href="mailto:--with-ssl-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../libressl" target="_blank">
--with-ssl-dir=/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/../libressl</a> --with-openmp=true --with-threadsafety=true --with-log=0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #1 PetscInitialize_DynamicLibraries() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/dll/reg.c:88" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/dll/reg.c:88</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #2 PetscInitialize_Common() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/objects/pinit.c:1030" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/objects/pinit.c:1030</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #3 PetscInitialize() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/objects/pinit.c:1276" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/petsc/petsc-3.19.2/src/sys/objects/pinit.c:1276</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #4 SlepcInitialize() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/sys/slepcinit.c:264" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/sys/slepcinit.c:264</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #5 EPSCreate() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/eps/interface/epsbasic.c:56" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/eps/interface/epsbasic.c:56</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: #6 EPSCreate() at <a href="mailto:/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/eps/interface/epsbasic.c:56" target="_blank">
/SCRATCH/tcad/ci_2_0/workspace/incremental_build_main@2/slepc/slepc-3.19.2/src/eps/interface/epsbasic.c:56</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Start calculating extreme eigenvalues..<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: ------------------------------------------------------------------------<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: or see<span class="m_4225664414938108325apple-converted-space"> </span><a href="https://urldefense.com/v3/__https:/petsc.org/release/faq/*valgrind__;Iw!!A4F2R9G_pg!fam7m9EAcUH3ksmrAcDKqw2b56d-5BMr-p7YSieNL7fFnEN9xuYzPbkyxhVcvvp16WSAXnrY-j895HpbMw$" target="_blank">https://petsc.org/release/faq/#valgrind</a><span class="m_4225664414938108325apple-converted-space"> </span>and<span class="m_4225664414938108325apple-converted-space"> </span><a href="https://urldefense.com/v3/__https:/petsc.org/release/faq/__;!!A4F2R9G_pg!fam7m9EAcUH3ksmrAcDKqw2b56d-5BMr-p7YSieNL7fFnEN9xuYzPbkyxhVcvvp16WSAXnrY-j8Q5wI00Q$" target="_blank">https://petsc.org/release/faq/</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: to get more information on the crash.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=59<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">system msg for write_line failure : Bad file descriptor<u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div></blockquote></div>