[Swift-devel] faster branch

[Lorenzo Pesce]

Mihael,

Sounds great.

Do you think that I can try and use it for medium scale simulations on Beagle as a test? (say 100 nodes, few thousands of concurrent jobs, just running for a couple of hours)

The current versions of swift just can't handle the workload on Beagle, or at least don't seem to, and your changes seem to fit perfectly with our needs.
Speed in handling complicated (or at least large) workflows and memory reductions are the weakest spot right now, so I am eager to try it.

If you give me the green light, I can try and install it and then run it.

It would be awesome because next week I need to get some fairly large (millions of tasks) workflows running.


On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote:

> Hi,
> 
> Over the past couple of weeks I committed the faster branch. It's
> essentially a re-implementation of the karajan language. 
> 
> The gist of it is that it is now a semi-compiled language. What I mean
> by that is that there is no code generation, so the initial parse tree
> is converted into a compiled tree. Variable and function references are
> resolved at compile-time. This provides considerable improvements in
> performance from two fronts:
> 
> 1. The interpreter was changed from an event-based system (a system that
> made sense many years ago, but has little relevance for swift) to a
> lightweight threading implementation based on exceptions.
> 2. The compiler can make optimizations that weren't possible with a
> dynamic language. For example, once the swift and logging settings are
> known, code that conditionally executes based on a swift property or
> logging statements can be optimized away and have no run-time overhead.
> 
> There is also a benefit in memory consumed. Variables are now stored in
> arrays rather than hashtables. This should reduce the run-time overhead
> of the engine.
> 
> There are a few more relevant changes from swift's perspective:
> - there is no more karajan xml. Keeping two inter-operable syntaxes was
> a pain in the back. 
> - sites.xml is now parsed with a DOM parser
> - all the coaster code has been moved into provider-coaster. Karajan is
> now independent of that (and the other way around).
> 
> Most of the tests pass, but that means little. So if you can give it a
> shot, please do. If it works sufficiently well, we can merge this into
> trunk.
> 
> Code is at:
> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/
> https://svn.ci.uchicago.edu/svn/vdl2/branches/faster
> 
> Mihael
> 
> (PS: Contrary to my initial beliefs, the TUI seems to mostly work, but I
> haven't tested in detail).
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel