[Swift-user] Running first.swift remotely on NCSA

[Andriy Fedorov]

>> In the qstat I see one job running on 1 node (the node type is NOT
>> "compute"), then it finishes, then Swift script reports "Final status:
>>  Finished successfully:1", and then I see SECOND job with 1 node
>> running in qstat.

> You should see a final job run after every successful Swift run, to clean
> up working files. So this second job does not seem incorrect - it isn't
> specified in your SwiftScript; instead you get it automatically.

I understand. I was not aware of this.

>> But it is just "echo"! Is this normal that it takes so long? When I do
>> the same test with WS GRAM and Fork, it completes in less than minute.
>
> Its a symptom of pre-ws gram + pbs + the way PBS is configured on TG UC.
>

Your explanation makes perfect sense! I assume, in order to avoid this
delay, I better figure out how to run everything with WS GRAM.

It's really nice to know these little details, so that I can
understand a little bit how it works under the hood.


>> The problem turned out to be in the setup of our firewall. For the
>> time being, I cannot resolve it, so I am trying to use TeraGrid UC
>> site for testing.

> Were you running the Swift command on a TG UC login node, submitting to TG
> UC?

Yes, correct.

>> $ swift first.swift
>> Unable to find required classes (javax.activation.DataHandler and
>> javax.mail.internet.MimeMultipart). Attachment support is disabled.
>> Swift 0.5 swift-r1783 cog-r1962

>> RunID: 20080620-1514-in8cf2yg
>> Progress:
>> echo_gt2_pbs started
>> Progress:  Stage in:1
>> Progress:  Executing:1
>> Progress:  Executing:1
>> ...

> The above looks more like something I'd expect from a WS-GRAM setup -
> specifically, the "Unable to find required classes..." message (which is
> safe to ignore here) and the symptom of going into "Executing" state,
> seeing it run in qstat but not detected so by Swift.

> Please can you run this one again and check that you really are submitting
> to pre-WS gram there?

Yes, it was submitted to pre-WS GRAM. I just ran it again, and it
behaves the same way, the only difference now is that I can explain
what is going on!

I think the first problem I had (running Swift with pre-WS+PBS) is
completely resolved, now that I understand the sources of those
delays.

I will do some testing for the second issue (passing GRAM parameters
for WS-GRAM+PBS), and prepare simple scripts and configuration files I
use, so that you could recreate the problem.

Thanks again for your help!

Andrey