[petsc-dev] Is there a good reason that BuildSystem's cuda.py requires GNU compilers?

[Balay, Satish]

On Tue, 19 Mar 2019, Mills, Richard Tran via petsc-dev wrote:

> Colleagues,
> 
> It took me a while to get PETSc to build at all with anything on Summit other than the GNU compilers, but, once this was accomplished, editing out the isGNU() test and then passing something like
> 
>     '--with-cuda=1',
>     '--with-cudac=nvcc -ccbin pgc++',

Does the following also work?

--with-cuda=1 --with-cudac=nvcc CUDAFLAGS='-ccbin pgc++'

Sometimes we have extra options in configure for specific features for
ex: --with-pic --with-visibility etc.

But that gets messy. On cuda side - we've have --with-cuda-arch and at
some point elimiated it [so CUDAFLAGS is now the interface for this
flag].  We could add --with-cuda-internal-compiler option to petsc
configure - but it will again have similar drawbacks. I personally
think most users will gravitate towards specifying such option via
CUDAFLAGS

> 
> to configure works fine. So, I should make a change to the BuildSystem cuda.py along these lines. I'm wondering exactly how I should make this work. I could just remove the check,

sure

> but I think that maybe the better thing to do is to check isGNU(), then if the compiler is *not* GNU, configure should add the appropriate '-ccbin' argument to "--with-cudac", unless the user has specified '-ccbin' in their '--with-cudac' already. Or do we need to get this fancy?

The check should be: do --compiler-options= constructed by  PETSc configure  work with CUDAC

[or perhaps we should - just trim the --compiler-options to only -I flags?]

I think we should avoid explict check for a compiler type [i.e isGNU() check] as much as possible.

> 
> CUDA is only supposed to work with certain compilers, but there doesn't seem to be a correct official list (for instance, it supposedly won't work with the IBM XL compilers, but they certainly *are* actually supported on Summit). Heck, the latest GCC suite won't even work right now. Since what compilers are supported seems to be in flux, I suggest we just let the user try anything and then let things fail if it doesn't work.

I suspec the list is dependent on the install [for ex: linux vs Windows vs somthing else?] and version of cuda [for ex: each version of cuda supports only specific versions of gcc]

Satish

> 
> --Richard
> 
> On 3/12/19 8:45 PM, Smith, Barry F. wrote:
> 
> 
>   Richard,
> 
>     You need to remove the isGNU() test and then experiment with getting the Nvidia tools to use the compiler you want it to use.
> 
>      No one has made a serious effort to use any other compilers but Gnu (at least not publicly).
> 
>    Barry
> 
> 
> 
> 
> 
> On Mar 12, 2019, at 10:40 PM, Mills, Richard Tran via petsc-dev <petsc-dev at mcs.anl.gov><mailto:petsc-dev at mcs.anl.gov> wrote:
> 
> Fellow PETSc developers,
> 
> If I try to configure PETSc with CUDA support on the ORNL Summit system using non-GNU compilers, I run into an error due to the following code in packages/cuda.py:
> 
>   def configureTypes(self):
>     import config.setCompilers
>     if not config.setCompilers.Configure.isGNU(self.setCompilers.CC, self.log):
>       raise RuntimeError('Must use GNU compilers with CUDA')
>   ...
> 
> Is this just because this code predates support for other host compilers with nvcc, or is there perhaps some more subtle reason that I, with my inexperience using CUDA, don't know about? I'm guessing that I just need to add support for using '-ccbin' appropriately to set the location of the non-GNU host compiler, but maybe there is something that I'm missing. I poked around in the petsc-dev mailing list archives and can find a few old threads on using non-GNU compilers, but I'm not sure what conclusions were reached.
> 
> Best regards,
> Richard
> 
> 
> 
> 
> 
> 
> 
>