<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
On 3/18/19 7:29 PM, Balay, Satish wrote:<br>
<blockquote type="cite" cite="mid:alpine.LFD.2.21.1903182118150.2404@sb">
<pre class="moz-quote-pre" wrap="">On Tue, 19 Mar 2019, Mills, Richard Tran via petsc-dev wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Colleagues,
It took me a while to get PETSc to build at all with anything on Summit other than the GNU compilers, but, once this was accomplished, editing out the isGNU() test and then passing something like
'--with-cuda=1',
'--with-cudac=nvcc -ccbin pgc++',
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
Does the following also work?
--with-cuda=1 --with-cudac=nvcc CUDAFLAGS='-ccbin pgc++'</pre>
</blockquote>
Yes, using CUDAFLAGS as above also works, and that does seem to be a better way to do things.<br>
<br>
After experimenting with a lot of different builds on Summit, and doing more reading about how CUDA compilation works on different platforms, I'm now thinking that perhaps configure.py should *avoid* doing anything clever to try figure out what the value of
"-ccbin" should be. For one, this is not anything that NVIDIA's toolchain does for the user in the first place: If you want to use nvcc with a host compiler that isn't whatever NVIDIA considers the default (g++ on Linux, clang on Mac OS, MSVC on Windows),
NVIDIA expects you to provide the appropriate '-ccbin' argument. Second, nvcc isn't the only CUDA compiler that a user might want to use: some people use Clang directly to compile CUDA code. Third, which host compilers are supported appears to be platform
independent; for example, GCC is the default/preferred host compiler on Linux, but isn't even supported on Mac OS! Figuring out what is supported is very convoluted, and I think that trying to get configure to determine this may be more trouble than it is
worth. I think we should instead let the user try whatever, and print out a helpful message how they "may need to specify host compiler to nvcc with -ccbin" if the CUDA compiler doesn't seem to work. Also, I'll put something about this in the CUDA configure
examples. Any objections?<br>
<br>
<blockquote type="cite" cite="mid:alpine.LFD.2.21.1903182118150.2404@sb">
<pre class="moz-quote-pre" wrap="">
Sometimes we have extra options in configure for specific features for
ex: --with-pic --with-visibility etc.
But that gets messy. On cuda side - we've have --with-cuda-arch and at
some point elimiated it [so CUDAFLAGS is now the interface for this
flag]. We could add --with-cuda-internal-compiler option to petsc
configure - but it will again have similar drawbacks. I personally
think most users will gravitate towards specifying such option via
CUDAFLAGS
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
to configure works fine. So, I should make a change to the BuildSystem cuda.py along these lines. I'm wondering exactly how I should make this work. I could just remove the check,
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
sure
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">but I think that maybe the better thing to do is to check isGNU(), then if the compiler is *not* GNU, configure should add the appropriate '-ccbin' argument to "--with-cudac", unless the user has specified '-ccbin' in their '--with-cudac' already. Or do we need to get this fancy?
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
The check should be: do --compiler-options= constructed by PETSc configure work with CUDAC</pre>
</blockquote>
Is there currently an existing check like this somewhere? Or will things just fail when running 'make' right now?<br>
<blockquote type="cite" cite="mid:alpine.LFD.2.21.1903182118150.2404@sb">
<pre class="moz-quote-pre" wrap="">
[or perhaps we should - just trim the --compiler-options to only -I flags?]
I think we should avoid explict check for a compiler type [i.e isGNU() check] as much as possible.
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
CUDA is only supposed to work with certain compilers, but there doesn't seem to be a correct official list (for instance, it supposedly won't work with the IBM XL compilers, but they certainly *are* actually supported on Summit). Heck, the latest GCC suite won't even work right now. Since what compilers are supported seems to be in flux, I suggest we just let the user try anything and then let things fail if it doesn't work.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
I suspec the list is dependent on the install [for ex: linux vs Windows vs somthing else?] and version of cuda [for ex: each version of cuda supports only specific versions of gcc]</pre>
</blockquote>
Yes, you are correct about this, as I detailed above.<br>
<blockquote type="cite" cite="mid:alpine.LFD.2.21.1903182118150.2404@sb">
<pre class="moz-quote-pre" wrap="">
Satish
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">
--Richard
On 3/12/19 8:45 PM, Smith, Barry F. wrote:
Richard,
You need to remove the isGNU() test and then experiment with getting the Nvidia tools to use the compiler you want it to use.
No one has made a serious effort to use any other compilers but Gnu (at least not publicly).
Barry
On Mar 12, 2019, at 10:40 PM, Mills, Richard Tran via petsc-dev <a class="moz-txt-link-rfc2396E" href="mailto:petsc-dev@mcs.anl.gov"><petsc-dev@mcs.anl.gov></a><a class="moz-txt-link-rfc2396E" href="mailto:petsc-dev@mcs.anl.gov"><mailto:petsc-dev@mcs.anl.gov></a> wrote:
Fellow PETSc developers,
If I try to configure PETSc with CUDA support on the ORNL Summit system using non-GNU compilers, I run into an error due to the following code in packages/cuda.py:
def configureTypes(self):
import config.setCompilers
if not config.setCompilers.Configure.isGNU(self.setCompilers.CC, self.log):
raise RuntimeError('Must use GNU compilers with CUDA')
...
Is this just because this code predates support for other host compilers with nvcc, or is there perhaps some more subtle reason that I, with my inexperience using CUDA, don't know about? I'm guessing that I just need to add support for using '-ccbin' appropriately to set the location of the non-GNU host compiler, but maybe there is something that I'm missing. I poked around in the petsc-dev mailing list archives and can find a few old threads on using non-GNU compilers, but I'm not sure what conclusions were reached.
Best regards,
Richard
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
</pre>
</blockquote>
<br>
</body>
</html>