[petsc-dev] PETSc-Cuda with gfortran 4.7

[Karl Rupp]

Hi Dave,

 > OK.  Just wanted to make sure I was not hijacking the list for an 
unintended purpose.

no, not at all. I highly appreciate the input you provided on resolving 
this nvcc/gcc hodgepodge. :-)

Best regards,
Karli


 >
> ________________________________________
> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Karl Rupp [rupp at mcs.anl.gov]
> Sent: Friday, March 15, 2013 10:42 AM
> To: petsc-dev at mcs.anl.gov
> Subject: Re: [petsc-dev] PETSc-Cuda with gfortran 4.7
>
> Hi Dave,
>
> my email was a remark for Asmund to get his code compiled. :-)
> Sorry if this wasn't clear enough...
>
> Best regards,
> Karli
>
>
> On 03/15/2013 11:15 AM, Nystrom, William D wrote:
>> Hi Karli,
>>
>> I'm not sure what point you are trying to make below.  As I indicated in my emails, I am using
>> 4.7.2 compilers for everything except nvcc and 4.7.2 is the default version of compilers for my
>> distro, Fedora 17.  However, nvcc constrains the version of compilers based on the Cuda
>> distribution.  Cuda 5.0 requires 4.6.x or earlier.  Cuda 4.2 requires 4.5.x or earlier, etc.  Perhaps
>> Cuda 5.1 will support 4.7.x but I have no idea when it will be released.  The procedure I shared
>> below allows one to restrict use of 4.6.x to only nvcc.
>>
>> Also, I don't have experience with other distros like Debian or Ubuntu.  So I can't speak with any
>> confidence on them although I don't see why the nvcc.profile approach would depend on the
>> distro.
>>
>> Also, I'm just trying to share my experiences with building petsc in a 4.7.2 compiler environment
>> to help Asmund.  If this is not appropriate for this mailing list, I'm happy to communicate directly
>> with Asmund.
>>
>> Best regards,
>>
>> Dave
>>
>> ________________________________________
>> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Karl Rupp [rupp at mcs.anl.gov]
>> Sent: Friday, March 15, 2013 9:43 AM
>> To: For users of the development version of PETSc
>> Cc: Åsmund Ervik
>> Subject: Re: [petsc-dev] PETSc-Cuda with gfortran 4.7
>>
>> Hi guys,
>>
>> depending on the distribution you are using, there may be multiple
>> versions of GCC already in the repository. For example, on Debian-based
>> distributions you can have multiple versions of GCC installed, which
>> show up in the package manager as gcc-4.6, gcc-4.7, etc.
>>
>> Ubuntu for example even allows you to switch between the versions
>> seamlessly using
>>     $> sudo update-alternatives --config gcc
>> An example can be found here:
>> http://askubuntu.com/questions/182027/how-to-use-gcc-4-7-in-place-of-4-6-ubuntu-12-04
>>
>> Aside from that, it is still not satisfactory for us to keep
>> recommending other compilers. GCC 4.7 will soon be the default on many
>> distributions (if it isn't already)...
>>
>> Best regards,
>> Karli
>>
>>
>>
>> On 03/15/2013 10:36 AM, Nystrom, William D wrote:
>>> Hi Asmund,
>>>
>>> In petsc-dev, files with a .cu extension are compiled with nvcc.
>>>
>>> I also tried the suggestion of soft linking an acceptable version of gcc/g++ into
>>> /usr/local/cuda/bin.  I think I used a gcc/g++ from a compatibility package that
>>> I installed with yum.  That did not work for me either.  I think the issue might
>>> have been related to that version being in the ccache directory but am not sure.
>>> The only solution I got to work was to install a new, acceptable version of the
>>> GNU compilers by actually building from source and installing in its own location
>>> such as somewhere in your home directory or in some place like /usr/local or
>>> /opt.  Also, if you use the "--program-suffix=46" configure option, I think you
>>> may also need to softlink gcc46 to gcc and g++46 to g++ in your install directory.
>>> I could check that when I get home this evening as well as checking my build
>>> script to make sure there is not something important in it to pass on.
>>>
>>> For me, it took about about an hour to build gcc 4.6.3, once I had learned the
>>> various things I passed on in the earlier email.  So you may have to just build
>>> a version of gcc rather than using a version from your distro's package system.
>>> BTW, what distro are you using?  I don't believe that nvcc uses gfortran.  It uses
>>> g++ and maybe also gcc.  Also, there is nothing special about gcc 4.6.3.  You
>>> could build an earlier version of gcc such as gcc 4.5.x or gcc 4.4.x.  I chose
>>> gcc 4.6.3 because it was the most up to date version that was supported by
>>> nvcc from the Cuda 5 distribution.
>>>
>>> I'll write more from home this evening when I can check some things on my
>>> system.
>>>
>>> Cheers,
>>>
>>> Dave
>>>
>>> ________________________________________
>>> From: Åsmund Ervik [Asmund.Ervik at sintef.no]
>>> Sent: Friday, March 15, 2013 7:21 AM
>>> To: petsc-dev at mcs.anl.gov; Nystrom, William D
>>> Subject: Re: [petsc-dev] PETSc-Cuda with gfortran 4.7
>>>
>>> Hi Dave,
>>>
>>> The errors occur when compiling these CUDA files:
>>> mpiaijAssemble.cu, aijcusp.cu, mpicusp.cu and veccusp.cu
>>> I assume these are compiled with nvcc, but I haven't triple checked that.
>>>
>>> I had not tried your trick with setting the C compiler in nvcc.profile.
>>> I did, however, try symlinking gcc-4.4 binaries into /usr/local/cuda/bin
>>> (as was suggested in one stackexchange post I found), but this produced
>>> an error during configure. Using your nvcc.profile, it gives the same error:
>>>
>>> CUDA version error: PETSC currently requires CUDA version 4.0 or higher
>>> - when compiling with CUDA
>>>
>>> Even though I have CUDA 5.0 installed. Just to be sure, are there
>>> binaries beside gfortran, gcc and g++ which must be found in the
>>> CUDA-specific gcc-4.6 folder? I have both gcc variants installed through
>>> the packaging system of my distro, so I cant't tell which belong to the
>>> 4.6 variant.
>>>
>>> Asmund
>>>
>>> On 13. mars 2013, wdn at lanl.gov wrote:
>>>>
>>>> Asmund,
>>>>
>>>> Do the failures you report below occur in cuda code that is ultimately compiled with nvcc?
>>>> I have not had problems compiling petsc-dev with cuda support on a Fedora 17 system
>>>> which uses gcc-4.7.2 and gfortran-4.7.2.  However, I have gcc 4.6.3 installed also and
>>>> have nvcc pointed at it by using the /usr/local/cuda/bin/nvcc.profile with a couple of lines
>>>> added.  So, when I build petsc-dev on my Fedora 17 system, it uses gcc-4.7.2 and
>>>> gfortran-4.7.2 for everything except cuda code located in .cu files.  I would worry about
>>>> removing some functionality in 4.7.2 with your suggested modification.
>>>>
>>>> Thanks,
>>>>
>>>> Dave
>>>>
>>>> --
>>>> Dave Nystrom
>>>> LANL HPC-5
>>>> Phone: 505-667-7913
>>>> Email: wdn at lanl.gov
>>>> Smail: Mail Stop B272
>>>>           Group HPC-5
>>>>           Los Alamos National Laboratory
>>>>           Los Alamos, NM 87545
>>>>
>>>>
>>
>