[Swift-devel] coaster problem with jobmanager=gt2:pbs

[Mihael Hategan]

Oops. cog r2367 should fix that.

On Mon, 2009-04-06 at 15:17 -0500, Michael Wilde wrote:
> Mihael, I just updated our test swift+cog source and rebuilt.
> 
> Glen is now getting:
> 
> Caused by:
>          Invalid GSSCredentials
> org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: 
> Invalid GSSCredentials
>          at 
> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submitSingleJob(JobSubmissionTaskHandler.java:149)
>          at 
> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submit(JobSubmissionTaskHandler.java:99)
>          at 
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:46)
>          at 
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:50)
>          at 
> org.globus.cog.abstraction.coaster.service.job.manager.WorkerManager.startWorker(WorkerManager.java:222)
>          at 
> org.globus.cog.abstraction.coaster.service.job.manager.WorkerManager.run(WorkerManager.java:145)
> Caused by: GSSException: Invalid name provided [Caused by: [JGLOBUS-112] 
> Malformed name, "=" missing in "38356/jobmanager-pbs"]
>          at 
> org.globus.gsi.gssapi.GlobusGSSName.<init>(GlobusGSSName.java:137)
>          at 
> org.globus.gsi.gssapi.GlobusGSSManagerImpl.createName(GlobusGSSManagerImpl.java:304)
>          at 
> org.globus.gsi.gssapi.auth.IdentityAuthorization.getExpectedName(IdentityAuthorization.java:82)
>          at org.globus.gram.Gram.gatekeeperConnect(Gram.java:85)
>          at org.globus.gram.Gram.request(Gram.java:310)
>          at org.globus.gram.GramJob.request(GramJob.java:262)
>          at 
> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submitSingleJob(JobSubmissionTaskHandler.java:133)
>          ... 5 more
> what's up here
> 
> 
> Any chance I picked up code in transition, or a new problem in recent 
> commits?
> 
> - Mike
> 
> 
> 
> On 4/6/09 1:02 PM, Mihael Hategan wrote:
> > Yes. This is one of those "can't find executable unless run through
> > 'bash -l' or maybe not" which we saw using wget and md5sum. I'm thinking
> > how to deal with the situation.
> > 
> > On Mon, 2009-04-06 at 12:10 -0500, Michael Wilde wrote:
> >> With this sites entry:
> >>
> >> <pool handle="qb" >
> >>    <profile namespace="globus" key="project">TG-CDA070002T</profile>
> >>    <execution provider="coaster" url="queenbee.loni-lsu.teragrid.org"
> >>               jobManager="gt2:pbs" />
> >>    <gridftp url="gsiftp://qb1.loni.org"/>
> >>    <workdirectory>/home/ux454325/swiftwork</workdirectory>
> >> </pool>
> >>
> >> I get the error below. Files are on CI net at /home/wilde/swift/lab.
> >>
> >> I will try to copy coaster boot logs and gram logs to same place when I 
> >> find them, in subdirs named by $RunID.logs.
> >>
> >> --
> >>
> >> com$ swift -tc.file tc.data -sites.file qb.coasters.xml cat.swift
> >> Swift svn swift-r2809 cog-r2350
> >>
> >> RunID: 20090406-1155-pgc5nj00
> >> Progress:
> >> Progress:  Stage in:1
> >> Progress:  Submitted:1
> >> Failed to transfer wrapper log from cat-20090406-1155-pgc5nj00/info/m on qb
> >> Progress:  Failed:1
> >> Execution failed:
> >>          Exception in cat:
> >> Arguments: [data.txt]
> >> Host: qb
> >> Directory: cat-20090406-1155-pgc5nj00/jobs/m/cat-m91ku09j
> >> stderr.txt:
> >>
> >> stdout.txt:
> >>
> >> ----
> >>
> >> Caused by:
> >>          Cannot submit job: Cannot run program "qsub": 
> >> java.io.IOException: error=2, No such file or directory
> >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: 
> >> Cannot submit job: Cannot run program "qsub": java.io.IOException: 
> >> error=2, No such file or directory
> >>          at 
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> >>          at 
> >> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:46)
> >>          at 
> >> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:50)
> >>          at 
> >> org.globus.cog.abstraction.coaster.service.job.manager.WorkerManager.startWorker(WorkerManager.java:221)
> >>          at 
> >> org.globus.cog.abstraction.coaster.service.job.manager.WorkerManager.run(WorkerManager.java:145)
> >> Caused by: java.io.IOException: Cannot run program "qsub": 
> >> java.io.IOException: error=2, No such file or directory
> >>          at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> >>          at java.lang.Runtime.exec(Runtime.java:593)
> >>          at 
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:73)
> >>          at 
> >> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> >>          ... 4 more
> >> Caused by: java.io.IOException: java.io.IOException: error=2, No such 
> >> file or directory
> >>          at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> >>          at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> >>          at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> >>          ... 7 more
> >>
> >> Cleaning up...
> >> Shutting down service at https://208.100.92.21:44166
> >> Got channel MetaChannel: 24235184 -> GSSSChannel-null(1)
> >> - Done
> >> com$ pwd
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >