[Swift-devel] Re: Does hookClass sites tag affect concurrency/throttling?

[Mihael Hategan]

It should not, because that call happens in a thread separate from the
job submission one, but while we're at it, it should also return as soon
as possible (such that the block management thread doesn't get stuck
doing stuff it wasn't supposed to do).

Mihael

On Mon, 2010-12-06 at 07:35 -0600, Michael Wilde wrote:
> Note that I'm not certain that the hookClass is indeed the cause of the throttling.
> 
> - Mike
> 
> ----- Original Message -----
> > This is a callback that is triggered by coasters when a Block is
> > allocated. This result is a real surprise, I'll check this out right
> > away.
> > 
> > On Sun, 5 Dec 2010, Michael Wilde wrote:
> > 
> > > Justin, what does this line in sites.xml do on the BG/P?
> > >
> > >    <profile namespace="globus"
> > >    key="hookClass">org.globus.swift.data.policy.AllocationHook</profile>
> > >
> > > I think I got that from one of your examples for BG/P use; when I
> > > use
> > > it, I seem to get throttled to about 20 active jobs max. When I take
> > > it
> > > off, I seem to be able to utilize all CPUs in a pset (256 cores in
> > > my
> > > last test).
> > >
> > > My full pool element which seems to be limiting me to about 20
> > > active is
> > > this (despite the fact that the throttle should let 276 jobs run at
> > > once):
> > >
> > >  <pool handle="surveyor-pecos-2">
> > >    <filesystem provider="local" />
> > >    <execution provider="coaster-persistent"
> > >    url="http://localhost:1985" jobmanager=""/>
> > >    <profile namespace="globus" key="workerManager">passive</profile>
> > >    <!-- profile namespace="globus"
> > >    key="internalHostname">172.17.3.16</profile> -->
> > >    <profile namespace="globus"
> > >    key="project">HTCScienceApps</profile>
> > >    <profile namespace="globus" key="queue">default</profile>
> > >    <profile namespace="globus" key="kernelprofile">zeptoos</profile>
> > >    <profile namespace="globus" key="alcfbgpnat">true</profile>
> > >    <profile namespace="karajan" key="jobthrottle">2.75</profile>
> > >    <profile namespace="karajan" key="initialScore">10000</profile>
> > >    <profile namespace="globus" key="workersPerNode">4</profile>
> > >    <profile namespace="globus" key="slots">1</profile>
> > >    <profile namespace="globus" key="maxTime">3600</profile>
> > >    <profile namespace="globus" key="nodeGranularity">64</profile>
> > >    <profile namespace="globus" key="maxNodes">64</profile>
> > >    <profile namespace="globus"
> > >    key="hookClass">org.globus.swift.data.policy.AllocationHook</profile>
> > >    <scratch>/dev/shm</scratch>
> > >    <workdirectory>$rundir</workdirectory>
> > >  </pool>
> > >
> > > When I use the following pool element, I get the full expected
> > > concurrency:
> > >
> > >  <pool handle="surveyor-pecos">
> > >    <execution provider="coaster-persistent"
> > >    url="http://localhost:1985" jobmanager=""/>
> > >    <profile namespace="globus" key="workerManager">passive</profile>
> > >
> > >    <profile namespace="globus" key="workersPerNode">4</profile>
> > >    <profile key="jobThrottle" namespace="karajan">3.00</profile>
> > >    <profile namespace="karajan" key="initialScore">10000</profile>
> > >
> > >    <filesystem provider="local" url="none" />
> > >    <scratch>/dev/shm</scratch>
> > >    <workdirectory>$rundir</workdirectory>
> > >  </pool>
> > >
> > > While I am not *sure* its the hookClass tag thats causing the
> > > throttling, but its my primary suspect. I will try to confirm, but
> > > Im still curious what this tag does.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > >
> > 
> > --
> > Justin M Wozniak
>