[Swift-devel] Coaster Task Submission Stalling

[Mihael Hategan]

Hi Tim,

I've never seen this before with pure Java.

Do you have logs from these runs?

Mihael

On Wed, 2014-09-03 at 16:49 -0500, Tim Armstrong wrote:
> I'm running a test Swift/T script that submit tasks to Coasters through the
> C++ client and I'm seeing some odd behaviour where task
> submission/execution is stalling for ~2 minute periods.  For example, I'm
> seeing submit log messages like "submitting
> urn:133-1409778135377-1409778135378: /bin/hostname" in bursts of several
> seconds with a gap of roughly 2 minutes in between, e.g. I'm seeing bursts
> with the following intervals in my logs.
> 
> 16:07:04,603 to 16:07:10,391
> 16:09:07,377 to 16:09:13,076
> 16:11:10,005 to 16:11:16,770
> 16:13:13,291 to 16:13:19,296
> 16:15:16,000 to 16:15:21,602
> 
> From what I can tell, the delay is on the coaster service side: the C
> client is just waiting for a response.
> 
> The jobs are just being submitted through the local job manager, so I
> wouldn't expect any delays there.  The tasks are also just "/bin/hostname",
> so should return immediately.
> 
> I'm going to continue digging into this on my own, but the 2 minute delay
> seems like a big clue: does anyone have an idea what could cause stalls in
> task submission of 2 minute duration?
> 
> Cheers,
> Tim