[Swift-devel] changes

[Michael Wilde]

Tim, there's the Site Guide which covers this in a per-site manner:

http://swift-lang.org/guides/trunk/siteguide/siteguide.html

We need to revise this material and integrate more of the concepts into 
the User Guide.

- Mike

On 7/4/14, 11:49 AM, Tim Armstrong wrote:
> I had a semi-related question: is there any documentation aside from 
> the user guide about sites, providers, and job managers in Swift/Coasters?
>
> - Tim
>
>
> On Fri, Jul 4, 2014 at 11:02 AM, Michael Wilde <wilde at anl.gov 
> <mailto:wilde at anl.gov>> wrote:
>
>     A lot of great work, here, Mihael - very nice!
>
>     We'll need to re-adjust the new config mechanism to match this,
>     unless you've already done that.
>
>     This means that we'll need to get all users converted to the new
>     config before this gets released - else they will have no
>     backwards compatibility for their current configurations (sites
>     and tc files).
>
>     Nonetheless, as we discussed in a prior meeting, I think we agreed
>     that this was a necessary step and the right way to go.
>
>     - Mike
>
>
>     On 7/4/14, 3:07 AM, Mihael Hategan wrote:
>>     Hi,
>>
>>     I committed a fairly large number of changes. Some things got limited
>>     testing, and some are work in progress. For example, I need to update
>>     sites files used in tests, but it's getting late here.
>>     Anyway, here's a summary:
>>
>>     1. New sites.xml format and removal of tc.data. I expect a bunch of
>>     things to break, but we discussed cleaning both sites.xml and tc.data
>>     many times and I guess I never got the courage to do it until now. This
>>     format is not backwards compatible (intentionally). It goes like this:
>>     <sites>
>>        <apps>? <!-- global apps -->
>>          <property name="...">...</property>* <!-- global app properties -->
>>          <env name="...">...</env>* <!-- global app envs -->
>>          <app name="..." executable="...">*
>>             <property/>*
>>             <env/>*
>>          </app>
>>        </apps>
>>
>>        <site name="...">
>>          <execution provider="..."...>
>>            <property/>*
>>          </execution>
>>          <filesystem>?
>>           <property/>*
>>          </filesystem>
>>          <wokdirectory/>
>>          <scratch/>?
>>          <apps/>?
>>        <site>
>>     </sites>
>>
>>     There are no more namespaces since they were mostly used to figure out
>>     whether a property was going to the task or the site. I also changed
>>     "profile" to "property", since "profile" seemed a bit arcane.
>>
>>     You should also now be able to use maxParallelTasks and
>>     initialParallelTasks instead of jobThrottle and initialScore. It
>>     computes the latter automatically.
>>
>>     2. Coasters now support multiple configurations. So you can have two
>>     local:local coaster sites with different settings and it should work.
>>     The way this works is that every configuration (and every different run)
>>     gets its own job queue, settings, and block allocator, and they don't
>>     interact with each other. This might be problematic with things like
>>     slots which is meant to limit the number of jobs globally, but that's a
>>     minor annoyance.
>>
>>     3. Passive workers can be launched with a -c(oncurrency) argument (1 by
>>     default). This will be the jobs per node setting. Each worker can have a
>>     different number, and that should work as expected. For passive workers,
>>     the client side jobsPerNode setting will be ignored.
>>
>>     4. There is a tool to analyze logs now (swift-log-info). This is simply
>>     an offline version of the http monitor. It can parse logs and feed the
>>     information to a web browser. It should also be able to follow logs as
>>     they are produced and feed live information to a browser. It can also
>>     fake a live log by parsing it slowly instead of all-at-once.
>>
>>     5. The http monitor got a bit of an update. You can look at all kinds of
>>     statistics, browse through apps, look at pretty plots of how apps
>>     behaved, etc. all in a nice dynamically updated ajaxy and hyperlinked
>>     fashion. It should be cute and useful in one. Won't work with older
>>     logs. Some pics attached.
>>
>>     6. There are nowworker.pl  <http://worker.pl>  "probes" for CPU usage, disk usage, and some
>>     I/O stats. If you click on a link to a worker in the http monitor,
>>     you'll get nice graphs of these things. The probes are currently
>>     hardcoded to run every 60 seconds (although in the pictures below they
>>     ran at 1/s).
>>
>>     Mihael
>>
>>
>>     _______________________________________________
>>     Swift-devel mailing list
>>     Swift-devel at ci.uchicago.edu  <mailto:Swift-devel at ci.uchicago.edu>
>>     https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
>     -- 
>     Michael Wilde
>     Mathematics and Computer Science          Computation Institute
>     Argonne National Laboratory               The University of Chicago
>
>
>     _______________________________________________
>     Swift-devel mailing list
>     Swift-devel at ci.uchicago.edu <mailto:Swift-devel at ci.uchicago.edu>
>     https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
>

-- 
Michael Wilde
Mathematics and Computer Science          Computation Institute
Argonne National Laboratory               The University of Chicago

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20140704/5f7d8029/attachment.html>