[petsc-dev] PETSc with CUDA on windows with VS2013 and CUDA 6.5

[Satish Balay]

On Fri, 5 Sep 2014, Paul Mullowney wrote:

> Hi,
> 
> I've been trying to build PETSc in cygwin with Visual Studio 2013 and CUDA
> 6.5. I successfully built and ran the PETSc GPU code following the
> instructions for building on cygwin. However I ran in to a few problems
> along the way. If anyone has insight into the second problem, I would
> appreciate it.
> 
> 1) cuda.py (lines 112, 113):
> if not config.setCompilers.Configure.isGNU(self.setCompilers.CC):
>   raise RuntimeError('Must use GNU compilers with CUDA')
> 
> I had to comment these lines out to enable configuration success.

We might have had issues on linux with folks trying --with-cc=icc and
--with-cudac=nvcc [and nvcc internally using gcc causing conflit].

So I think this check was added at that time. Ideally it should query
nvcc to see what the internal compiler is - and then check if
'--with-cc' is compatible with it. But I don't know how to do that.

Perhaps the easier fix is either of:

if (isLinux() and not isGNU(cc)) then error

if (not isWindows() and not isGNU()) then error

> 
> 2) This may be related to why the GNU restriction in 1) was put in place.
> Whenever PETSc has to compile a .cu, it fails. For example for a single
> threaded build, the first .cu file to be compiled is veccusp.cu. For a
> verbose build, I see:
> 
> /home/Paul/projects/petsc/bin/win32fe/win32fe nvcc -c -O -arch=sm_30
>  --compiler-options="-MT -wd4996 -O2   -I/home/Paul/projects/petsc/include
> -I/home/Paul/projects/petsc/cuda_build/include
> -I/cygdrive/c/CUDA_6.5/include -I/home/Paul/projects/cusp/
> -I/home/Paul/projects/cusp/include -I/cygdrive/c/CUDA_6.5/include/thrust/
> -I/home/Paul/projects/petsc/include/mpiuni   "
> /home/Paul/projects/petsc/src/vec/vec/impls/seq/seqcusp/veccusp.cu -o
> cuda_build/obj/src/vec/vec/impls/seq/seqcusp/veccusp.o # Compile first so
> that if there is an error, it comes from a normal compile
> 
> veccusp.cu
> *nvcc fatal   : redefinition of argument 'optimize'*
> 
> 
> However, if I copy and paste the the compile line above and run it in the
> terminal, it succeeds. If you do this for every file .cu file in PETSc
> (~10-15 files). PETSc will build and work properly on GPUs in cygwin.
> 
> I don't know what the problem is here. Insight from anyone with
> windows/cygwin/CUDA experience would be much appreciated. I've attached a
> configure.log and make.log.

For one - I haven't tried cuda on windows in a while. [It must be when
win32fe was modified to support nvcc - 3 years back]

And I don't unserstand the error. Is it complaining about '-O' vs '-O2'?

And the fact that copy/paste works is strange.

You can try: 'make all-legacy' and see if that works.

Or you can try Using --verbose option i.e 'win32fe nvcc --verbose' to
see exactly what win32fe is doing [from gmake and from direct
invocation] - and see if there are differences.

Perhaps there is a 'make' variable [thats read-in as env variable by
nvcc] thats causing nvcc to behave differently.

Satish