<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1">
<div style="direction: ltr;font-family: Arial;color: #000000;font-size: 14pt;">Mark,<br>
<br>
I did not say that "threads are discouraged". Rather, I said that it was<br>
"my understanding" that use of the "current" thread implementation in<br>
PETSc was discouraged because a decision had been made to do a<br>
redesign and reimplementation of the PETSc thread support package.<br>
And that has certainly been consistent with my attempts to test out<br>
the threadcomm package and report issues that I found. In particular,<br>
I did a lot of testing and benchmarking of the threadcomm package<br>
back in the summer of 2013 and provided extensive feedback to the<br>
PETSc Team on my findings. I received very little comment from the<br>
PETSc Team on my feedback except from Karli. By the end of August,<br>
2013 I was told that the threadcomm package was going to be redesigned<br>
and rewritten and that there was not any interest in fixing the issues<br>
that I had encountered and reported with the current version of<br>
threadcomm. And, since then, I have not been able to detect any<br>
work on the current threadcomm package. I have requested updates<br>
from Jed a couple of times since 2013 and his replies have seemed<br>
consistent with the plan to redesign and rewrite the PETSc thread<br>
support. So, I'm just trying to learn what the latest plan and schedule<br>
for PETSc thread support is because I have access to platforms where<br>
I think it could be very useful. But without the issues I reported in 2013<br>
being fixed, the current thread support is not very useful to me.<br>
<br>
Thanks,<br>
<br>
<div>Dave<br>
<br>
<div style="font-family:Tahoma; font-size:13px"><font size="2"><span style="font-size:10pt">--
<br>
Dave Nystrom<br>
LANL HPC-5<br>
Phone: 505-667-7913<br>
Email: wdn@lanl.gov<br>
Smail: Mail Stop B272<br>
Group HPC-5<br>
Los Alamos National Laboratory<br>
Los Alamos, NM 87545<br>
</span></font><br>
</div>
</div>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF880701"><font color="#000000" face="Tahoma" size="2"><b>From:</b> Mark Adams [mfadams@lbl.gov]<br>
<b>Sent:</b> Friday, January 09, 2015 11:35 AM<br>
<b>To:</b> Nystrom, William David<br>
<b>Cc:</b> Jed Brown; Barry Smith; For users of the development version of PETSc<br>
<b>Subject:</b> Re: [petsc-dev] PETSc and threads<br>
</font><br>
</div>
<div></div>
<div>
<div dir="ltr">Dave,
<div><br>
<div>Why do you think threads are discouraged? PETSc tends to not keep dead code around so if it is in the repo its "supported" with the caveat that resources are not infinite.
<div><br>
</div>
</div>
<div>BTW, I am using threads with hypre and gamg on Titan. Not sure it is helping yet but the solves (not setup) are fully threaded AFAIK.</div>
<div><br>
</div>
<div>Mark</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jan 9, 2015 at 11:59 AM, Nystrom, William David <span dir="ltr">
<<a href="mailto:wdn@lanl.gov" target="_blank">wdn@lanl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
So is there any schedule for the availability of the new PETSc thread model implementation?<br>
My understanding is that the current thread implementation in PETSc is not even supported<br>
by the PETSc Team and use of it is discouraged. I'm interested in this capability for both<br>
Sequoia and Trinity and have been thinking about making a PETSc interface to one of the<br>
main LANL ASC codes.<br>
<br>
Dave<br>
<br>
--<br>
Dave Nystrom<br>
LANL HPC-5<br>
Phone: <a href="tel:505-667-7913" value="+15056677913" target="_blank">505-667-7913</a><br>
Email: <a href="mailto:wdn@lanl.gov" target="_blank">wdn@lanl.gov</a><br>
Smail: Mail Stop B272<br>
Group HPC-5<br>
Los Alamos National Laboratory<br>
Los Alamos, NM 87545<br>
<br>
<br>
________________________________________<br>
From: <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] on behalf of Jed Brown [<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>]<br>
Sent: Friday, January 09, 2015 8:44 AM<br>
To: Mark Adams; Barry Smith<br>
Cc: For users of the development version of PETSc<br>
Subject: Re: [petsc-dev] PETSc and threads<br>
<div class="HOEnZb">
<div class="h5"><br>
Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> writes:<br>
> No this is me. They will probably have about 30K (2D linear FE) equations<br>
> per 40 Tflop node. 10% (4 Tflops) is too much resources for 30K equations<br>
> as it is. No need to try utilize the GPU as far as I can see.<br>
<br>
With multiple POWER9 sockets per node, you have to deal with NUMA and<br>
separate caches. The rest of the application is not going to do this<br>
with threads, so you'll have multiple MPI processes anyway. The entire<br>
problem will fit readily in L2 cache and you have a latency problem on<br>
the CPU alone. Ask them to make neighborhood collectives fast.<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</body>
</html>