<div dir="ltr">FYI, I was running the test incorrectly:<div>03:38 cgpu12 ~/petsc_install$ srun -n 1 -G 1 ./a.out <br>70<br>70<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 26, 2021 at 10:21 PM Mark Adams <<a href="mailto:mfadams@lbl.gov">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I had git bisect working and was 4 steps away when I got a new crash.<div>configure.log is empty.<br><div><br><div>19:15 1 cgpu02 (a531cba26b...)|BISECTING ~/petsc$ git bisect bad<br>Bisecting: 19 revisions left to test after this (roughly 4 steps)<br>[149e269f455574fbe8ce3ebaf42121ae7fdf0635] Merge branch 'tisaac/feature-spqr' into 'main'<br>19:16 cgpu02 (149e269f45...)|BISECTING ~/petsc$ ../arch-cori-gpu-opt-gcc.py PETSC_DIR=$PWD<br>===============================================================================<br> Configuring PETSc to compile on your system <br>===============================================================================<br>*******************************************************************************<br> CONFIGURATION CRASH (Please send configure.log to <a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a>)<br>*******************************************************************************<br><br>EOL while scanning string literal (cuda.py, line 176)<br> File "/global/u2/m/madams/petsc/config/configure.py", line 455, in petsc_configure<br> framework = config.framework.Framework(['--configModules=PETSc.Configure','--optionsModule=config.compilerOptions']+sys.argv[1:], loadArgDB = 0)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 107, in __init__<br> self.createChildren()<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 344, in createChildren<br> self.getChild(moduleName)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 329, in getChild<br> config.setupDependencies(self)<br> File "/global/u2/m/madams/petsc/config/PETSc/Configure.py", line 80, in setupDependencies<br> self.blasLapack = framework.require('config.packages.BlasLapack',self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 349, in require<br> config = self.getChild(moduleName, keywordArgs)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 329, in getChild<br> config.setupDependencies(self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/packages/BlasLapack.py", line 21, in setupDependencies<br> config.package.Package.setupDependencies(self, framework)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/package.py", line 151, in setupDependencies<br> self.mpi = framework.require('config.packages.MPI',self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 349, in require<br> config = self.getChild(moduleName, keywordArgs)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 329, in getChild<br> config.setupDependencies(self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/packages/MPI.py", line 73, in setupDependencies<br> self.mpich = framework.require('config.packages.MPICH', self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 349, in require<br> config = self.getChild(moduleName, keywordArgs)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 329, in getChild<br> config.setupDependencies(self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/packages/MPICH.py", line 16, in setupDependencies<br> self.cuda = framework.require('config.packages.cuda',self)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 349, in require<br> config = self.getChild(moduleName, keywordArgs)<br> File "/global/u2/m/madams/petsc/config/BuildSystem/config/framework.py", line 302, in getChild<br> type = __import__(moduleName, globals(), locals(), ['Configure']).Configure<br>19:16 cgpu02 (149e269f45...)|BISECTING ~/petsc$ ../arch-cori-gpu-opt-gcc.py PETSC_DIR=$PWD<br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 26, 2021 at 10:10 PM Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 26, 2021 at 6:13 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto"><div><br></div><div> What is HOST=cori09 Does it have GPUs?</div><div><br></div> <a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6" target="_blank">https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6</a><div><br></div><div> Seems to clearly state</div><div><br></div><div><dl><dt style="background-color:rgb(239,239,240);border-top:1px solid rgb(216,220,222);border-bottom:1px solid rgb(238,243,245);padding:3px;font-family:"Trebuchet MS","DIN Pro",sans-serif;font-size:14px">int <a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp" title="" shape="rect" style="color:rgb(118,185,0)" target="_blank">cudaDeviceProp</a>::<a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6" shape="rect" style="color:rgb(118,185,0)" target="_blank">major</a> [inherited] </dt><dd style="margin-left:20px;margin-bottom:20px;font-family:"Trebuchet MS","DIN Pro",sans-serif;font-size:14px"><div style="margin-top:0.8em;margin-bottom:2em"><p style="margin-left:0px;margin-top:0px">Major compute capability </p><div><br></div><div><br></div><div>Mark, please compile and run this program on the machine you are running configure on</div><div><br></div><div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><stdio.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(105,103,149)"><span style="font-variant-ligatures:no-common-ligatures">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)"><cuda.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_runtime.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_runtime_api.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_device_runtime_api.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(45,150,30)"><span style="font-variant-ligatures:no-common-ligatures">int</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(74,0,255)">main</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">(</span><span style="font-variant-ligatures:no-common-ligatures">int</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">arg</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">,</span><span style="font-variant-ligatures:no-common-ligatures">char</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> **</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">args</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">)</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">{</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(45,150,30)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(194,0,255)">struct</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures">cudaDeviceProp</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">dp</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">;</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> cudaGetDeviceProperties(&dp, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> printf(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)">"%d\n"</span><span style="font-variant-ligatures:no-common-ligatures">,10*dp.major+dp.minor);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(45,150,30)">int</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">major</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">minor</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> printf(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)">"%d\n"</span><span style="font-variant-ligatures:no-common-ligatures">,10*major+minor);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(194,0,255)">return</span><span style="font-variant-ligatures:no-common-ligatures">(0);</span></div></div></div></dd></dl></div></div></div></blockquote><div>Probably, you need to check the return code of these two function calls to make sure they are correct.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto"><div><dl><dd style="margin-left:20px;margin-bottom:20px;font-family:"Trebuchet MS","DIN Pro",sans-serif;font-size:14px"><div style="margin-top:0.8em;margin-bottom:2em"><div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">}</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></div></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px">This is what I get </div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><br></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">$ nvcc mytest.c -lcuda</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">~/petsc</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(200,20,201)"><b> (main=)</b></span><span style="font-variant-ligatures:no-common-ligatures"> arch-main</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">$ ./a.out</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">70</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">70</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;font-stretch:normal;line-height:normal">Which is exactly what it is suppose to do.</div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div>Barry</div></div></div></dd></dl><div><br><blockquote type="cite"><div>On May 26, 2021, at 5:31 PM, Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:</div><br><div><div><br> Yes, this code which I guess never got hit before <br><br>cudaDeviceProp dp; cudaGetDeviceProperties(&dp, 0); printf("%d\n",10*dp.major+dp.minor); return(0);;<br><br>is using the wrong property for the generation. <br><br> Back to the CUDA documentation for the correct information. <br><br><br><br><blockquote type="cite">On May 26, 2021, at 3:47 PM, Jacob Faibussowitsch <<a href="mailto:jacob.fai@gmail.com" target="_blank">jacob.fai@gmail.com</a>> wrote:<br><br>1120 sounds suspiciously like some CUDA version rather than architecture or compute capability…<br><br>Best regards,<br><br>Jacob Faibussowitsch<br>(Jacob Fai - booss - oh - vitch)<br>Cell: +1 (312) 694-3391<br><br><blockquote type="cite">On May 26, 2021, at 22:29, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br><br>I started to get this error today on Cori. <br><br>nvcc fatal : Unsupported gpu architecture 'compute_1120'<br><br>I am pretty sure I had a clean build but I can redo it if you don't know where this is from.<br><br>Thanks,<br>Mark<br><configure.log><br></blockquote></blockquote><br></div></div></blockquote></div><br></div></div></div></blockquote></div></div>
</blockquote></div>
</blockquote></div>