<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 26, 2021 at 7:13 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto"><div><br></div><div>  What is HOST=cori09  Does it have GPUs?</div></div></div></blockquote><div><br></div><div>That is the login node that I got the interactive compute node from. This is my node:</div><div><br></div><div> SLURM_JOB_NODELIST=cgpu02</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto"><div><br></div>  <a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6" target="_blank">https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6</a><div><br></div><div>  Seems to clearly state</div><div><br></div><div><dl><dt style="background-color:rgb(239,239,240);border-top:1px solid rgb(216,220,222);border-bottom:1px solid rgb(238,243,245);padding:3px;font-family:"Trebuchet MS","DIN Pro",sans-serif;font-size:14px">int  <a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp" title="" shape="rect" style="color:rgb(118,185,0)" target="_blank">cudaDeviceProp</a>::<a href="https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp_164490976c8e07e028a8f1ce1f5cd42d6" shape="rect" style="color:rgb(118,185,0)" target="_blank">major</a> [inherited] </dt><dd style="margin-left:20px;margin-bottom:20px;font-family:"Trebuchet MS","DIN Pro",sans-serif;font-size:14px"><div style="margin-top:0.8em;margin-bottom:2em"><p style="margin-left:0px;margin-top:0px">Major compute capability </p><div><br></div><div><br></div><div>Mark, please compile and run this program on the machine you are running configure on</div><div><br></div><div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><stdio.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(105,103,149)"><span style="font-variant-ligatures:no-common-ligatures">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)"><cuda.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_runtime.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_runtime_api.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(157,32,111)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(105,103,149)">#include</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures"><cuda_device_runtime_api.h></span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(45,150,30)"><span style="font-variant-ligatures:no-common-ligatures">int</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(74,0,255)">main</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">(</span><span style="font-variant-ligatures:no-common-ligatures">int</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">arg</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">,</span><span style="font-variant-ligatures:no-common-ligatures">char</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> **</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">args</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">)</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">{</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;color:rgb(45,150,30)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(194,0,255)">struct</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures">cudaDeviceProp</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">dp</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">;</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">                cudaGetDeviceProperties(&dp, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">                printf(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)">"%d\n"</span><span style="font-variant-ligatures:no-common-ligatures">,10*dp.major+dp.minor);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">                </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(45,150,30)">int</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">major</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(193,101,28)">minor</span><span style="font-variant-ligatures:no-common-ligatures">;</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap">         </span>cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, 0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">                printf(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(157,32,111)">"%d\n"</span><span style="font-variant-ligatures:no-common-ligatures">,10*major+minor);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">                </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(194,0,255)">return</span><span style="font-variant-ligatures:no-common-ligatures">(0);</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">}</span></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></div></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px">This is what I get </div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><br></div><div style="margin:0px;font-stretch:normal;line-height:normal;font-family:Menlo;min-height:16px"><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">$ nvcc mytest.c -lcuda</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">~/petsc</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(200,20,201)"><b> (main=)</b></span><span style="font-variant-ligatures:no-common-ligatures"> arch-main</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">$ ./a.out</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">70</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">70</span></div><div style="margin:0px;font-stretch:normal;line-height:normal"><br></div></div></div></dd></dl></div></div></div></blockquote><div>This compiled and ran fine, but the output is wrong:</div><div> </div><div>18:04 cgpu02  ~/petsc_install$ ./a.out <br>1120<br>-1431545180</div><div><br></div><div>It looks to me like there was a regression. I am running git bisect now. 7 more steps.</div><div><br></div></div></div>