[petsc-users] odd behavior when using lapack's dgeev with petsc

Sat Apr 8 23:31:32 CDT 2017

It was a library from a colleague, particularly for internal use. So, this will not perpetuate to your users. I appreciate your concern and support. 

We will fix this at our end. 

-Manav

> On Apr 8, 2017, at 11:28 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> 
>> On Apr 8, 2017, at 11:21 PM, Manav Bhatia <bhatiamanav at gmail.com> wrote:
>> 
>> It turned out to be another third-party library that had some functions names same as those from BLAS.  
> 
>   Can you tell us what library it was? We can add some checks for this and thus prevent others from having to spend so much time, like you had to, figure it out. IMHO it is evil for a library to include routines with the same names as in BLAS and they should fix their libraries.
> 
>> This was confusing LAPACK. 
>> Building without that library is leading to expected results from the code. 
>> 
>> What I don’t understand is why this shows up on one platform out of the four different OSs that I have been using this code on. 
> 
> 
>  It depends on the order that the OS references the libraries, so this is actually not surprising.
> 
>  Barry
> 
> 
>> 
>> -Manav
>> 
>>> On Apr 8, 2017, at 2:20 AM, Satish Balay <balay at mcs.anl.gov> wrote:
>>> 
>>> I would do a minimal petsc build - without any packages from
>>> /usr/local - and see if the problem presists..
>>> 
>>> Satish
>>> 
>>> On Fri, 7 Apr 2017, Barry Smith wrote:
>>> 
>>>> 
>>>>> On Apr 7, 2017, at 3:34 PM, Manav Bhatia <bhatiamanav at gmail.com> wrote:
>>>>> 
>>>>> Yes, I printed the data in both cases and they look the same. 
>>>>> 
>>>>> I also used “set step-mode on” to show the system lapack info, and they both are using the same lapack routine. 
>>>>> 
>>>>> This is still baffling me. 
>>>> 
>>>> alignment of the input arrays, both the same? 
>>>> 
>>>> I don't know why this is happening; what if you use your standalone code but link against all the libraries that are linked against for the PETSc case.
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> -Manav
>>>>> 
>>>>> 
>>>>>> On Apr 7, 2017, at 3:22 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Apr 7, 2017, at 2:57 PM, Manav Bhatia <bhatiamanav at gmail.com> wrote:
>>>>>>> 
>>>>>>> Hi Barry, 
>>>>>>> 
>>>>>>> Thanks for the inputs. 
>>>>>>> 
>>>>>>> I did try that, but the debugger (gdb) stepped right over the dgeev_ call, without getting inside the function. 
>>>>>> 
>>>>>> Did it at least stop at the function so you do an up and print all the arguments passed in?
>>>>>> 
>>>>>>> 
>>>>>>> I am wondering if this has anything to do with the fact that the system lapack library might not have any debugging info in it. 
>>>>>> 
>>>>>> Yeah I forgot it might not have them.
>>>>>> 
>>>>>> Barry
>>>>>> 
>>>>>>> 
>>>>>>> Thoughts? 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Manav
>>>>>>> 
>>>>>>>> On Apr 7, 2017, at 2:40 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Apr 7, 2017, at 1:46 PM, Manav Bhatia <bhatiamanav at gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi, 
>>>>>>>>> 
>>>>>>>>> I have compile petsc on my Ubuntu machine (also Mac OS 10.12 separately) to link to the system lapack and blas libraries (shown below). 
>>>>>>>>> 
>>>>>>>>> I have created an interface class to dgeev in lapack to calculate the eigenvalues of a matrix. 
>>>>>>>>> 
>>>>>>>>> My application code links to multiple libraries: libMesh, petsc, slepc, hdf5, etc. 
>>>>>>>>> 
>>>>>>>>> If I test my interface inside this application code, I get junk results. 
>>>>>>>> 
>>>>>>>> This is easy to debug because you have a version that works.
>>>>>>>> 
>>>>>>>> Run both versions in separate windows each in a debugger and put a break point in the dgeev_ function. When it gets there check that it is the same dgeev_ function in both cases and check that the inputs are the same then step through both to see when things start to change between the two.
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> However, on the same machine, if I use the interface in a separate main() function without linking to any of the libraries except lapack and blas, then I get expected results. 
>>>>>>>>> 
>>>>>>>>> Also, this problem does not show up on Mac. 
>>>>>>>>> 
>>>>>>>>> I am not sure what could be causing this and don’t quite know where to start. Could Petsc have anything to do with this? 
>>>>>>>>> 
>>>>>>>>> Any insight would be greatly appreciated. 
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Manav
>>>>>>>>> 
>>>>>>>>> manav at manav1:~/test$ ldd /opt/local/lib/libpetsc.so
>>>>>>>>> 	linux-vdso.so.1 =>  (0x00007fff3e7a8000)
>>>>>>>>> 	libsuperlu_dist.so.5 => /opt/local/lib/libsuperlu_dist.so.5 (0x00007f721fbd1000)
>>>>>>>>> 	libparmetis.so => /opt/local/lib/libparmetis.so (0x00007f721f990000)
>>>>>>>>> 	libmetis.so => /opt/local/lib/libmetis.so (0x00007f721f718000)
>>>>>>>>> 	libsuperlu.so.5 => /opt/local/lib/libsuperlu.so.5 (0x00007f721f4a7000)
>>>>>>>>> 	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f721f124000)
>>>>>>>>> 	liblapack.so.3 => /usr/lib/liblapack.so.3 (0x00007f721e92c000)
>>>>>>>>> 	libblas.so.3 => /usr/lib/libblas.so.3 (0x00007f721e6bd000)
>>>>>>>>> 	libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f721e382000)
>>>>>>>>> 	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f721e079000)
>>>>>>>>> 	libmpi_mpifh.so.12 => /usr/lib/libmpi_mpifh.so.12 (0x00007f721de20000)
>>>>>>>>> 	libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f721daf4000)
>>>>>>>>> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f721d8f0000)
>>>>>>>>> 	libmpi.so.12 => /usr/lib/libmpi.so.12 (0x00007f721d61a000)
>>>>>>>>> 	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f721d403000)
>>>>>>>>> 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f721d1e6000)
>>>>>>>>> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f721ce1d000)
>>>>>>>>> 	/lib64/ld-linux-x86-64.so.2 (0x000055d739f1b000)
>>>>>>>>> 	libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f721cbcf000)
>>>>>>>>> 	libopen-pal.so.13 => /usr/lib/libopen-pal.so.13 (0x00007f721c932000)
>>>>>>>>> 	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f721c6f2000)
>>>>>>>>> 	libibverbs.so.1 => /usr/lib/libibverbs.so.1 (0x00007f721c4e3000)
>>>>>>>>> 	libopen-rte.so.12 => /usr/lib/libopen-rte.so.12 (0x00007f721c269000)
>>>>>>>>> 	libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f721c064000)
>>>>>>>>> 	libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f721be5e000)
>>>>>>>>> 	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f721bc56000)
>>>>>>>>> 	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f721ba52000)
>>>>>>>>> 	libhwloc.so.5 => /usr/lib/x86_64-linux-gnu/libhwloc.so.5 (0x00007f721b818000)
>>>>>>>>> 	libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f721b60c000)
>>>>>>>>> 	libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f721b402000)
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
> 