[Swift-devel] cleanup fails on abe

[Sarah Kenny]

On Fri, Aug 13, 2010 at 1:16 PM, Michael Wilde <wilde at mcs.anl.gov> wrote:
> Sarah, what does "shut off cleanup" mean?

sitedir.keep=true


>
> Jon, is there any similarity between what Sarah is encountering and what you observed on TeraPort (presumably using provider=coaster jobmanager=local:pbs)?
>
> - Mike
>
> ----- "Sarah Kenny" <skenny at uchicago.edu> wrote:
>
>> hi all, not sure if anyone else is running on abe, but for some
>> reason
>> cleanup seems to fail on there very consistently. swift throws a
>> warning:
>>
>> The following warnings have occurred:
>> 1. Cleanup on ABE failed
>> Caused by:
>>
>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>> Cannot submit job
>>         at
>> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submitSingleJob(JobSubmissionTaskHandler.java:146)
>>         at
>> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submit(JobSubmissionTaskHandler.java:100)
>>         at
>> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:46)
>>         at
>> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:50)
>>         at
>> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
>> Caused by: org.globus.gram.GramException: Parameter not supported
>>         at org.globus.gram.Gram.request(Gram.java:358)
>>         at org.globus.gram.GramJob.request(GramJob.java:262)
>>         at
>> org.globus.cog.abstraction.impl.execution.gt2.JobSubmissionTaskHandler.submitSingleJob(JobSubmissionTaskHandler.java:134)
>>         ... 4 more
>>
>> if i shut off cleanup, i don't get the warning and the workflow
>> 'apprears' to have completed successfully, however even with cleanup
>> shut off pbs still generates the email below giving the error:
>>
>>
>> i'm still poking around to see if i can figure out what's up, but
>> thought i would throw this out there in case someone else has come
>> across it.
>>
>> swift, coaster and gram logs attached.
>>
>> ~sk
>>
>> ---------- Forwarded message ----------
>> From: adm <adm at ncsa.uiuc.edu>
>> Date: Fri, Aug 13, 2010 at 12:53 PM
>> Subject: PBS JOB 3000582.abem5.ncsa.uiuc.edu
>> To: skenny at abe1196.ncsa.uiuc.edu
>>
>>
>> PBS Job Id: 3000582.abem5.ncsa.uiuc.edu
>> Job Name:   configtester
>> Exec host:
>>  abe0553/0+abe0314/0+abe0313/0+abe0311/0+abe0310/0+abe0307/0+abe0294/0+abe0290/0+abe0287/0+abe0286/0+abe0285/0+abe0284/0+abe0283/0+abe0279/0+abe0278/0+abe0277/0+abe0275/0+abe0273/0+abe0272/0+abe0271/0+abe0256/0+abe0254/0+abe0174/0+abe0173/0+abe0166/0+abe0165/0+abe0163/0+abe0087/0+abe0085/0+abe0084/0+abe0010/0+abe0387/0
>> An error has occurred processing your job, see below.
>> Post job file processing error; job 3000582.abem5.ncsa.uiuc.edu on
>> host
>> abe0553/0+abe0314/0+abe0313/0+abe0311/0+abe0310/0+abe0307/0+abe0294/0+abe0290/0+abe0287/0+abe0286/0+abe0285/0+abe0284/0+abe0283/0+abe0279/0+abe0278/0+abe0277/0+abe0275/0+abe0273/0+abe0272/0+abe0271/0+abe0256/0+abe0254/0+abe0174/0+abe0173/0+abe0166/0+abe0165/0+abe0163/0+abe0087/0+abe0085/0+abe0084/0+abe0010/0+abe0387/0
>>
>> Unable to copy file
>> /u/ac/skenny/.pbs_spool//3000582.abem5.ncsa.uiuc.edu.OU to
>> /u/ac/skenny/.globus/job/abe1196.ncsa.uiuc.edu/15575.1281721892/stdout
>> *** error from copy
>> /bin/cp: cannot create regular file
>> `/u/ac/skenny/.globus/job/abe1196.ncsa.uiuc.edu/15575.1281721892/stdout':
>> No such file or directory
>> *** end error output
>>
>> Unable to copy file
>> /u/ac/skenny/.pbs_spool//3000582.abem5.ncsa.uiuc.edu.ER to
>> /u/ac/skenny/.globus/job/abe1196.ncsa.uiuc.edu/15575.1281721892/stderr
>> *** error from copy
>> /bin/cp: cannot create regular file
>> `/u/ac/skenny/.globus/job/abe1196.ncsa.uiuc.edu/15575.1281721892/stderr':
>> No such file or directory
>> *** end error output
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
> --
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
>
>