<div dir="ltr"><div dir="ltr">On Thu, Aug 22, 2019 at 8:45 PM Lailai Zhu via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thank you guys, after i remove the system's metis-related things,<br>
it compiles well and go through the check. however when i try to<br>
use the elemental solver, it still does not work out. i basically get<br>
the segmentation fault error, which does not appear when i use<br>
the standard dense matrix and jacobi sovers. thanks in advance,<br>
<br>
best,<br>
lailai<br>
<br>
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, <br>
probably memory access out of range<br>
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
[0]PETSC ERROR: or see <br>
<a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a><br>
[0]PETSC ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS <br>
X to find memory corruption errors<br>
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, <br>
and run<br>
[0]PETSC ERROR: to get more information on the crash.<br></blockquote><div><br></div><div>It would be really helpful to get a stack trace from the debugger.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
On 8/22/19 8:03 PM, Smith, Barry F. wrote:<br>
><br>
>> On Aug 22, 2019, at 6:48 PM, Balay, Satish <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
>><br>
>> Compilers are supposed to prefer libraries in specified -L path before system stuff.<br>
> Suppose to.<br>
><br>
>> balay@es^~ $ ls /usr/lib/lib*metis*<br>
>> /usr/lib/libmetis.a /usr/lib/libmetis.so.3.1 /usr/lib/libparmetis.so@ /usr/lib/<a href="http://libscotchmetis-5.1.so" rel="noreferrer" target="_blank">libscotchmetis-5.1.so</a> /usr/lib/libscotchmetis.so@<br>
>> /usr/lib/libmetis.so@ /usr/lib/libparmetis.a /usr/lib/libparmetis.so.3.1 /usr/lib/libscotchmetis.a<br>
>> balay@es^~ $<br>
> This is really bad system management, there is no reason for them to be there nor should they be there.<br>
><br>
> bsmith@es:~$ ldd /usr/lib/libparmetis.so.3.1<br>
> linux-vdso.so.1 => (0x00007fff9115e000)<br>
> libmpi.so.1 => /usr/lib/libmpi.so.1 (0x00007faf95f87000)<br>
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007faf95c81000)<br>
> libmetis.so.3.1 => /usr/lib/libmetis.so.3.1 (0x00007faf95a34000)<br>
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faf9566b000)<br>
> libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007faf95468000)<br>
> libhwloc.so.5 => /usr/lib/x86_64-linux-gnu/libhwloc.so.5 (0x00007faf95228000)<br>
> libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007faf9501e000)<br>
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faf94e00000)<br>
> /lib64/ld-linux-x86-64.so.2 (0x00007faf9654e000)<br>
> libnuma.so.1 => /soft/com/packages/pgi/19.3/linux86-64/19.3/lib/libnuma.so.1 (0x00007faf94bf5000)<br>
> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faf949f1000)<br>
><br>
> You have something in /usr/lib referring to something in /soft/com/packages/pgi/ ??<br>
><br>
> and of course that refers back to<br>
><br>
> bsmith@es:~$ ls -l /soft/com/packages/pgi/19.3/linux86-64/19.3/lib/libnuma.so.1<br>
> lrwxrwxrwx 1 fritz voice 38 Mar 29 15:59 /soft/com/packages/pgi/19.3/linux86-64/19.3/lib/libnuma.so.1 -> /usr/lib/x86_64-linux-gnu/libnuma.so.1<br>
><br>
> I stand by my statement, it is bad policy to put any stuff like this in system directories.<br>
><br>
><br>
>> <<<<<br>
>><br>
>> And we have these files installed and they don't cause problems. And its not always practical to uninstall system stuff<br>
>> [esp on multi-user machines]<br>
> I agree it is not always practical or possible to remove them.<br>
><br>
> barry<br>
><br>
>> Satish<br>
>><br>
>><br>
>> On Thu, 22 Aug 2019, Smith, Barry F. wrote:<br>
>><br>
>>> You have a copy of parmetis installed in /usr/lib this is a systems directory and many compilers and linkers automatically find libraries in that location and it is often difficult to avoid have the compilers/linkers use these. In general you never want to install external software such as parmetis, PETSc, MPI, etc in systems directories (/usr/ and /usr/local)<br>
>>><br>
>>> You should delete this library (and the includes in /usr/include)<br>
>>><br>
>>> Barry<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>>> On Aug 22, 2019, at 5:17 PM, Balay, Satish via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
>>>><br>
>>>><br>
>>>>> ./ex19: symbol lookup error: /usr/lib/libparmetis.so: undefined symbol: ompi_mpi_comm_world<br>
>>>> For some reason the wrong parmetis library is getting picked up. I don't know why.<br>
>>>><br>
>>>> Can you copy/paste the log from the following?<br>
>>>><br>
>>>> cd src/snes/examples/tutorials<br>
>>>> make PETSC_DIR=/home/lailai/nonroot/petsc/petsc3.11.3_intel19_mpich3.3 ex19<br>
>>>> ldd ex19<br>
>>>><br>
>>>> cd /home/lailai/nonroot/petsc/petsc3.11.3_intel19_mpich3.3/pet3.11.3-intel19-mpich3.3/lib<br>
>>>> ldd *.so<br>
>>>><br>
>>>> Satish<br>
>>>><br>
>>>> On Thu, 22 Aug 2019, Lailai Zhu via petsc-users wrote:<br>
>>>><br>
>>>>> hi, Satish,<br>
>>>>><br>
>>>>> as you have suggested, i compiled a new version using 3.11.3,<br>
>>>>> it compiles well, the errors occur in checking. i also attach<br>
>>>>> the errors of check. thanks very much,<br>
>>>>><br>
>>>>> lailai<br>
>>>>><br>
>>>>> On 8/22/19 4:16 PM, Balay, Satish wrote:<br>
>>>>>> Any reason for using petsc-3.10.5 and not latest petsc-3.11?<br>
>>>>>><br>
>>>>>> I suggest starting from scatch and rebuilding.<br>
>>>>>><br>
>>>>>> And if you still have issues - send corresponding configure.log and make.log<br>
>>>>>><br>
>>>>>> Satish<br>
>>>>>><br>
>>>>>> On Thu, 22 Aug 2019, Lailai Zhu via petsc-users wrote:<br>
>>>>>><br>
>>>>>>> sorry, Satish,<br>
>>>>>>><br>
>>>>>>> but it does not seem to solve the problem.<br>
>>>>>>><br>
>>>>>>> best,<br>
>>>>>>> lailai<br>
>>>>>>><br>
>>>>>>> On 8/22/19 12:41 AM, Balay, Satish wrote:<br>
>>>>>>>> Can you run 'make' again and see if this error goes away?<br>
>>>>>>>><br>
>>>>>>>> Satish<br>
>>>>>>>><br>
>>>>>>>> On Wed, 21 Aug 2019, Lailai Zhu via petsc-users wrote:<br>
>>>>>>>><br>
>>>>>>>>> hi, Satish,<br>
>>>>>>>>> i tried to do it following your suggestion, i get the following errors<br>
>>>>>>>>> when<br>
>>>>>>>>> installing.<br>
>>>>>>>>> here is my configuration,<br>
>>>>>>>>><br>
>>>>>>>>> any ideas?<br>
>>>>>>>>><br>
>>>>>>>>> best,<br>
>>>>>>>>> lailai<br>
>>>>>>>>><br>
>>>>>>>>> ./config/configure.py --with-c++-support --known-mpi-shared-libraries=1<br>
>>>>>>>>> --with-batch=0 --with-mpi=1 --with-debugging=0 CXXOPTFLAGS="-g -O3"<br>
>>>>>>>>> COPTFLAGS="-O3 -ip -axCORE-AVX2 -xSSE4.2" FOPTFLAGS="-O3 -ip -axCORE-AVX2<br>
>>>>>>>>> -xSSE4.2" --with-blas-lapack-dir=/opt/intel/mkl --download-elemental=1<br>
>>>>>>>>> --download-blacs=1 --download-scalapack=1 --download-hypre=1<br>
>>>>>>>>> --download-plapack=1 --with-cc=mpicc --with-cxx=mpic++ --with-fc=mpifort<br>
>>>>>>>>> --download-amd=1 --download-anamod=1 --download-blopex=1<br>
>>>>>>>>> --download-dscpack=1 --download-sprng=1 --download-superlu=1<br>
>>>>>>>>> --with-cxx-dialect=C++11 --download-metis --download-parmetis<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> pet3.10.5-intel19-mpich3.3/obj/mat/impls/sbaij/seq/sbaij.o: In function<br>
>>>>>>>>> `MatCreate_SeqSBAIJ':<br>
>>>>>>>>> sbaij.c:(.text+0x1bc45): undefined reference to<br>
>>>>>>>>> `MatConvert_SeqSBAIJ_Elemental'<br>
>>>>>>>>> ld: pet3.10.5-intel19-mpich3.3/obj/mat/impls/sbaij/seq/sbaij.o:<br>
>>>>>>>>> relocation<br>
>>>>>>>>> R_X86_64_PC32 against undefined hidden symbol<br>
>>>>>>>>> `MatConvert_SeqSBAIJ_Elemental'<br>
>>>>>>>>> can not be used when making a shared object<br>
>>>>>>>>> ld: final link failed: Bad value<br>
>>>>>>>>> gmakefile:86: recipe for target<br>
>>>>>>>>> 'pet3.10.5-intel19-mpich3.3/lib/libpetsc.so.3.10.5' failed<br>
>>>>>>>>> make[2]: *** [pet3.10.5-intel19-mpich3.3/lib/libpetsc.so.3.10.5] Error 1<br>
>>>>>>>>> make[2]: Leaving directory<br>
>>>>>>>>> '/usr/nonroot/petsc/petsc3.10.5_intel19_mpich3.3'<br>
>>>>>>>>> ........................../petsc3.10.5_intel19_mpich3.3/lib/petsc/conf/rules:81:<br>
>>>>>>>>> recipe for target 'gnumake' failed<br>
>>>>>>>>> make[1]: *** [gnumake] Error 2<br>
>>>>>>>>> make[1]: Leaving directory<br>
>>>>>>>>> '/usr/nonroot/petsc/petsc3.10.5_intel19_mpich3.3'<br>
>>>>>>>>> **************************ERROR*************************************<br>
>>>>>>>>> Error during compile, check<br>
>>>>>>>>> pet3.10.5-intel19-mpich3.3/lib/petsc/conf/make.log<br>
>>>>>>>>> Send it and pet3.10.5-intel19-mpich3.3/lib/petsc/conf/configure.log to<br>
>>>>>>>>> <a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a><br>
>>>>>>>>><br>
>>>>>>>>> On 8/21/19 10:58 PM, Balay, Satish wrote:<br>
>>>>>>>>>> To install elemental - you use: --download-elemental=1 [not<br>
>>>>>>>>>> --download-elemental-commit=v0.87.7]<br>
>>>>>>>>>><br>
>>>>>>>>>> Satish<br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>> On Wed, 21 Aug 2019, Lailai Zhu via petsc-users wrote:<br>
>>>>>>>>>><br>
>>>>>>>>>>> hi, dear petsc developers,<br>
>>>>>>>>>>><br>
>>>>>>>>>>> I am having a problem when using the external solver elemental.<br>
>>>>>>>>>>> I installed petsc3.10.5 version with the flag<br>
>>>>>>>>>>> --download-elemental-commit=v0.87.7<br>
>>>>>>>>>>> the installation seems to be ok. However, it seems that i may not be<br>
>>>>>>>>>>> able<br>
>>>>>>>>>>> to use the elemental solver though.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> I followed this page<br>
>>>>>>>>>>> <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MATELEMENTAL.html" rel="noreferrer" target="_blank">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MATELEMENTAL.html</a><br>
>>>>>>>>>>> to interface the elemental solver, namely,<br>
>>>>>>>>>>> MatSetType(A,MATELEMENTAL);<br>
>>>>>>>>>>> or set it via the command line '*-mat_type elemental*',<br>
>>>>>>>>>>><br>
>>>>>>>>>>> in either case, i will get the following error,<br>
>>>>>>>>>>><br>
>>>>>>>>>>> [0]PETSC ERROR: --------------------- Error Message<br>
>>>>>>>>>>> --------------------------------------------------------------<br>
>>>>>>>>>>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing<br>
>>>>>>>>>>> package:<br>
>>>>>>>>>>> <a href="http://www.mcs.anl.gov/petsc/documentation/installation.html#external" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/installation.html#external</a><br>
>>>>>>>>>>> [0]PETSC ERROR: Unknown Mat type given: elemental<br>
>>>>>>>>>>> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><br>
>>>>>>>>>>> for<br>
>>>>>>>>>>> trouble shooting.<br>
>>>>>>>>>>> [0]PETSC ERROR: Petsc Release Version 3.10.5, Mar, 28, 2019<br>
>>>>>>>>>>><br>
>>>>>>>>>>> May i ask whether there will be a way or some specific petsc versions<br>
>>>>>>>>>>> that<br>
>>>>>>>>>>> are<br>
>>>>>>>>>>> able to use the elemental solver?<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Thanks in advance,<br>
>>>>>>>>>>><br>
>>>>>>>>>>> best,<br>
>>>>>>>>>>> lailai<br>
>>>>>>>>>>><br>
>>>>><br>
>>>>><br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>