<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Tim, there's the Site Guide which covers this in a per-site manner:<br>
<br>
<a class="moz-txt-link-freetext" href="http://swift-lang.org/guides/trunk/siteguide/siteguide.html">http://swift-lang.org/guides/trunk/siteguide/siteguide.html</a><br>
<br>
We need to revise this material and integrate more of the concepts
into the User Guide.<br>
<br>
- Mike<br>
<br>
<div class="moz-cite-prefix">On 7/4/14, 11:49 AM, Tim Armstrong
wrote:<br>
</div>
<blockquote
cite="mid:CAC0jiV68Uh2QxhRRYKvWSv7X5C_TKEt036Mrj2vf9E+Ex472ZQ@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div>I had a semi-related question: is there any documentation
aside from the user guide about sites, providers, and job
managers in Swift/Coasters?<br>
<br>
</div>
- Tim<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Jul 4, 2014 at 11:02 AM,
Michael Wilde <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:wilde@anl.gov" target="_blank">wilde@anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> A lot of great work,
here, Mihael - very nice!<br>
<br>
We'll need to re-adjust the new config mechanism to match
this, unless you've already done that.<br>
<br>
This means that we'll need to get all users converted to
the new config before this gets released - else they will
have no backwards compatibility for their current
configurations (sites and tc files).<br>
<br>
Nonetheless, as we discussed in a prior meeting, I think
we agreed that this was a necessary step and the right way
to go.<br>
<br>
- Mike
<div>
<div class="h5"><br>
<br>
<div>On 7/4/14, 3:07 AM, Mihael Hategan wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5">
<pre>Hi,
I committed a fairly large number of changes. Some things got limited
testing, and some are work in progress. For example, I need to update
sites files used in tests, but it's getting late here.
Anyway, here's a summary:
1. New sites.xml format and removal of tc.data. I expect a bunch of
things to break, but we discussed cleaning both sites.xml and tc.data
many times and I guess I never got the courage to do it until now. This
format is not backwards compatible (intentionally). It goes like this:
<sites>
<apps>? <!-- global apps -->
<property name="...">...</property>* <!-- global app properties -->
<env name="...">...</env>* <!-- global app envs -->
<app name="..." executable="...">*
<property/>*
<env/>*
</app>
</apps>
<site name="...">
<execution provider="..."...>
<property/>*
</execution>
<filesystem>?
<property/>*
</filesystem>
<wokdirectory/>
<scratch/>?
<apps/>?
<site>
</sites>
There are no more namespaces since they were mostly used to figure out
whether a property was going to the task or the site. I also changed
"profile" to "property", since "profile" seemed a bit arcane.
You should also now be able to use maxParallelTasks and
initialParallelTasks instead of jobThrottle and initialScore. It
computes the latter automatically.
2. Coasters now support multiple configurations. So you can have two
local:local coaster sites with different settings and it should work.
The way this works is that every configuration (and every different run)
gets its own job queue, settings, and block allocator, and they don't
interact with each other. This might be problematic with things like
slots which is meant to limit the number of jobs globally, but that's a
minor annoyance.
3. Passive workers can be launched with a -c(oncurrency) argument (1 by
default). This will be the jobs per node setting. Each worker can have a
different number, and that should work as expected. For passive workers,
the client side jobsPerNode setting will be ignored.
4. There is a tool to analyze logs now (swift-log-info). This is simply
an offline version of the http monitor. It can parse logs and feed the
information to a web browser. It should also be able to follow logs as
they are produced and feed live information to a browser. It can also
fake a live log by parsing it slowly instead of all-at-once.
5. The http monitor got a bit of an update. You can look at all kinds of
statistics, browse through apps, look at pretty plots of how apps
behaved, etc. all in a nice dynamically updated ajaxy and hyperlinked
fashion. It should be cute and useful in one. Won't work with older
logs. Some pics attached.
6. There are now <a moz-do-not-send="true" href="http://worker.pl" target="_blank">worker.pl</a> "probes" for CPU usage, disk usage, and some
I/O stats. If you click on a link to a worker in the http monitor,
you'll get nice graphs of these things. The probes are currently
hardcoded to run every 60 seconds (although in the pictures below they
ran at 1/s).
Mihael
</pre>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
Swift-devel mailing list
<a moz-do-not-send="true" href="mailto:Swift-devel@ci.uchicago.edu" target="_blank">Swift-devel@ci.uchicago.edu</a>
<a moz-do-not-send="true" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><span class="HOEnZb"><font color="#888888">
</font></span></pre>
<span class="HOEnZb"><font color="#888888"> </font></span></blockquote>
<span class="HOEnZb"><font color="#888888"> <br>
<pre cols="72">--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
</pre>
</font></span></div>
<br>
_______________________________________________<br>
Swift-devel mailing list<br>
<a moz-do-not-send="true"
href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>
<a moz-do-not-send="true"
href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel"
target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
</pre>
</body>
</html>