[petsc-users] Problem with NVIDIA compiler and OpenACC

[Barry Smith]

   Please send configure.log 

   We do not explicitly include libnvc but as Satish noted it may get listed when configure is generating link lines.

   With configure.log we'll know where it is being included (and we may be able to provide a fix that removes it  explicitly since it is apparently not needed according to the NVIDIA folks).

   Barry


> On Apr 4, 2024, at 10:33 AM, Frank Bramkamp <bramkamp at nsc.liu.se> wrote:
> 
> This Message Is From an External Sender 
> This message came from outside your organization.
> Thanks for the reply,
> 
> Do you know if you actively include the libnvc library ?!
> Or is this somehow automatically included ?! 
> 
> Greetings, Frank
> 
> 
> 
> 
>> On 4 Apr 2024, at 15:56, Satish Balay <balay at mcs.anl.gov <mailto:balay at mcs.anl.gov>> wrote:
>> 
>> 
>> On Thu, 4 Apr 2024, Frank Bramkamp wrote:
>> 
>>> Dear PETSC Team,
>>> 
>>> I found the following problem:
>>> I compile petsc 3.20.5 with Nvidia compiler 23.7.
>>> 
>>> 
>>> I use a pretty standard configuration, including
>>> 
>>> --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g" --with-debugging=0 --with-log=1 --download-fblaslapack --with-cuda=0
>>> 
>>> I exclude cuda, since I was not sure if the problem was cuda related. 
>> 
>> Can you try using (to exclude cuda): --with-cudac=0
>> 
>>> 
>>> 
>>> The problem is now, if I have s simple fortran program where I link the petsc library, but I actually do not use petsc in that program
>>> (Just for testing). I want to use OpenACC directives in my program, e.g. !$acc parallel loop .
>>> The problem is now, as soon I link with the petsc library, the openacc commands do not work anymore.
>>> It seems that openacc is not initialised and hence it cannot find a GPU.
>>> 
>>> The problem seems that you link with -lnvc.
>>> In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
>>> If I take this out, then openacc works. With “-lnvc” something gets messed up.
>>> 
>>> The problem is also discussed here:
>>> https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$<https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$ >
>>> 
>>> My understanding is that libnvc is more a runtime library that does not need to be included by the linker.
>>> Not sure if there is a specific reason to include libnvc (I am not so familiar what this library does).
>>> 
>>> If I take out -lnvc from “petscvariables”, then my program with openacc works as expected. I did not try any more realistic program that includes petsc.
>>> 
>>> 
>>> 
>>> 2)
>>> When compiling petsc with cuda support, I also found that in the petsc library the library libnvJitLink.so.12
>>> Is not found. On my system this library is in $CUDA_ROOT/lib64
>>> I am not sure where this library is on your system ?! 
>> 
>> Hm - good if you can send configure.log for this. configure attempts '$CC -v' to determine the link libraries to get c/c++/fortran compatibility libraries. But it can grab other libraries that the compilers are using internally here.
>> 
>> To avoid this - you can explicitly list these libraries to configure. For ex: for gcc/g++/gfortran
>> 
>> ./configure CC=gcc CXX=g++ FC=gfortran LIBS="-lgfortran -lstdc++"
>> 
>> Satish
>> 
>>> 
>>> 
>>> Thanks a lot, Frank Bramkamp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240404/45131b89/attachment-0001.html>