[Swift-devel] faster branch

[Lorenzo Pesce]

Thanks a lot. Sounds interesting and sensible. 

I will try to put it to work ASAP, perhaps even today (I have to hack a genomics pipeline together and that would be a good test).

We'll get swift use and running on Beagle squared up,  I am looking forward to runs zillions of jobs on tens of steps of intertwined workflows. :-)

BTW, today the second group claimed the name swift-seq for genomics analysis. We'll have a copyright internecine war...


On Feb 13, 2013, at 12:22 PM, Mihael Hategan wrote:

> On Wed, 2013-02-13 at 11:13 -0600, Lorenzo Pesce wrote:
>> Mihael,
>> 
>> Sounds great.
>> 
>> Do you think that I can try and use it for medium scale simulations on Beagle as a test? (say 100 nodes, few thousands of concurrent jobs, just running for a couple of hours)
>> 
>> The current versions of swift just can't handle the workload on Beagle, or at least don't seem to, and your changes seem to fit perfectly with our needs.
> 
> The change was motivated by your needs in part.
> 
>> Speed in handling complicated (or at least large) workflows and memory reductions are the weakest spot right now, so I am eager to try it.
>> 
>> If you give me the green light, I can try and install it and then run it.
> 
> Please, go ahead as long as we have the understanding that there is a
> higher than normal likelihood of problems.
> 
>> 
>> It would be awesome because next week I need to get some fairly large (millions of tasks) workflows running.
> 
> I can't promise this will work, but it might be worth a shot.
> 
> Mihael
> 
>