[Swift-user] Data transfer error

[Bronevetsky, Greg]

Ah, that makes a lot of sense! It might be useful to add an explicit check for this and a warning since my configuration ends up using just 20% of my available job allocations but tracking it down it non-trivial since poor efficiency may have many causes.

Greg Bronevetsky
Lawrence Livermore National Lab
(925) 424-5756
bronevetsky at llnl.gov
http://greg.bronevetsky.com


-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov] 
Sent: Wednesday, May 28, 2014 5:11 PM
To: Bronevetsky, Greg
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] Data transfer error

On Wed, 2014-05-28 at 23:48 +0000, Bronevetsky, Greg wrote:
> 	Are you specifying a max walltime for the apps?
> 
> 	If not, swift assumes 10 minutes. If the first few batches take 21 minutes, and the worker has 30 minutes allocated, it won't be able to fit any other jobs 	after that.
> My sites file has the following bounds:
>     <profile namespace="globus" key="maxtime">1800</profile>

We need to fix that one. So that's 30 minutes.

>     <profile namespace="globus" key="maxwalltime">00:24:00</profile>

And that's 24 minutes.

So if the worker is left with less than 24 minutes (i.e. after the first
6 minutes), none of the jobs will fit. You might want to lower the app maxwalltime to 10 minutes if that is the actual maximum time the app will take.

If 30 minutes is not a hard limit for the queue, increasing that should help.

Mihael