[Swift-devel] Re: Does hookClass sites tag affect concurrency/throttling?

[Michael Wilde]

Note that I'm not certain that the hookClass is indeed the cause of the throttling.

- Mike

----- Original Message -----
> This is a callback that is triggered by coasters when a Block is
> allocated. This result is a real surprise, I'll check this out right
> away.
> 
> On Sun, 5 Dec 2010, Michael Wilde wrote:
> 
> > Justin, what does this line in sites.xml do on the BG/P?
> >
> >    <profile namespace="globus"
> >    key="hookClass">org.globus.swift.data.policy.AllocationHook</profile>
> >
> > I think I got that from one of your examples for BG/P use; when I
> > use
> > it, I seem to get throttled to about 20 active jobs max. When I take
> > it
> > off, I seem to be able to utilize all CPUs in a pset (256 cores in
> > my
> > last test).
> >
> > My full pool element which seems to be limiting me to about 20
> > active is
> > this (despite the fact that the throttle should let 276 jobs run at
> > once):
> >
> >  <pool handle="surveyor-pecos-2">
> >    <filesystem provider="local" />
> >    <execution provider="coaster-persistent"
> >    url="http://localhost:1985" jobmanager=""/>
> >    <profile namespace="globus" key="workerManager">passive</profile>
> >    <!-- profile namespace="globus"
> >    key="internalHostname">172.17.3.16</profile> -->
> >    <profile namespace="globus"
> >    key="project">HTCScienceApps</profile>
> >    <profile namespace="globus" key="queue">default</profile>
> >    <profile namespace="globus" key="kernelprofile">zeptoos</profile>
> >    <profile namespace="globus" key="alcfbgpnat">true</profile>
> >    <profile namespace="karajan" key="jobthrottle">2.75</profile>
> >    <profile namespace="karajan" key="initialScore">10000</profile>
> >    <profile namespace="globus" key="workersPerNode">4</profile>
> >    <profile namespace="globus" key="slots">1</profile>
> >    <profile namespace="globus" key="maxTime">3600</profile>
> >    <profile namespace="globus" key="nodeGranularity">64</profile>
> >    <profile namespace="globus" key="maxNodes">64</profile>
> >    <profile namespace="globus"
> >    key="hookClass">org.globus.swift.data.policy.AllocationHook</profile>
> >    <scratch>/dev/shm</scratch>
> >    <workdirectory>$rundir</workdirectory>
> >  </pool>
> >
> > When I use the following pool element, I get the full expected
> > concurrency:
> >
> >  <pool handle="surveyor-pecos">
> >    <execution provider="coaster-persistent"
> >    url="http://localhost:1985" jobmanager=""/>
> >    <profile namespace="globus" key="workerManager">passive</profile>
> >
> >    <profile namespace="globus" key="workersPerNode">4</profile>
> >    <profile key="jobThrottle" namespace="karajan">3.00</profile>
> >    <profile namespace="karajan" key="initialScore">10000</profile>
> >
> >    <filesystem provider="local" url="none" />
> >    <scratch>/dev/shm</scratch>
> >    <workdirectory>$rundir</workdirectory>
> >  </pool>
> >
> > While I am not *sure* its the hookClass tag thats causing the
> > throttling, but its my primary suspect. I will try to confirm, but
> > Im still curious what this tag does.
> >
> > Thanks,
> >
> > Mike
> >
> >
> 
> --
> Justin M Wozniak

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory