<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Nov 3, 2012, at 8:20 PM, Mohammad Mirzadeh wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">Hum ... That did not help. I found this -threadcomm_affinities 0,1,2,3,4,5,6,7 but that also did nothing. Is that option any relevant?</div></blockquote><div>This option currently only works with pthreads.</div><blockquote type="cite"><div dir="ltr"><div><br></div><div>Another question: the linux box I'm testing on has 2 quad-cores (8 cores/2 sockets). If I run the code with mpirun -np 8, I get about 3X which makes sense to me. However, If I run with either pthread (which seems to use all cores) or openmp (which always defaults to 1 no matter what) </div></div></blockquote>As I said before you have to set the core affinity for OpenMP through environment variables.<br><blockquote type="cite"><div dir="ltr"><div>I get the same performance as serial. </div></div></blockquote>i) Which example are you running? Please send output of -log_summary.</div><div>ii) Only the vector and a few matrix operations are threaded currently. There are no threaded preconditioners yet.<br><blockquote type="cite"><div dir="ltr"><div>Does this mean there is something messed up with the hardware and/or how mpi/openmp/pthread is set up?</div></div></blockquote><div><br></div>Shri<br><blockquote type="cite"><div dir="ltr">
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Nov 2, 2012 at 8:24 PM, Shri <span dir="ltr"><<a href="mailto:abhyshr@mcs.anl.gov" target="_blank">abhyshr@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
On Nov 2, 2012, at 9:54 PM, Mohammad Mirzadeh wrote:<br>
<br>
> Hi,<br>
><br>
> To use the new thread model in PETSc, does it suffice to run the code with the following?<br>
><br>
> -threadcomm_type <openmp/pthread> -threadcomm_nthreads <#><br>
><br>
> When I run the code with openmp, only 1 processor/core is active (looking at top). When using pthread, all cores are active. Am I missing something?<br>
</div></div>OpenMP defaults to binding all threads to a single core if the cpu affinity is not specified explicitly. If you<br>
are using GNU OpenMP then you can bind the threads to specific cores using the environment variable GOMP_CPU_AFFINITY <a href="http://gcc.gnu.org/onlinedocs/libgomp/GOMP_005fCPU_005fAFFINITY.html" target="_blank">http://gcc.gnu.org/onlinedocs/libgomp/GOMP_005fCPU_005fAFFINITY.html</a><br>
If you are some other OpenMP implementation then check its manual to see how to set cpu affinity.<br>
<span class="HOEnZb"><font color="#888888"><br>
Shri<br>
<br>
</font></span></blockquote></div><br></div>
</blockquote></div><br></body></html>