[Swift-user] set cleanup off

[Mihael Hategan]

Hi Ketan,

There is no way currently to disable the cleanup job, unless you run
with provider staging, in which case there is no cleanup job.

In some sense, the below limitation of the queue is a way to disable the
cleanup job, and, apart from the distasteful error messages, there
should be no detrimental side-effects.

The harm in not doing cleanup is that you leave unneeded files on disk.
You can, of course, clean up the work directory manually when you
need/want to.

Mihael

On Fri, 2014-10-10 at 15:55 -0500, Ketan Maheshwari wrote:
> I am running cobalt jobs where I have a user quota of 1024 node runs max in
> an hour. In cases where this exceeds, the system will not allow any more
> job submission.
> 
> In this scenario, the cleanup operations fail after the run has completed
> with the following error:
> 
> Final status:Fri, 10 Oct 2014 20:49:58+0000  Finished successfully:100
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could
> not submit job (qsub reported an exit code of 1).
> project: ExM<Fault 1001: "The limit of 1024 node hours per user in the
> 'default' queue has been reached\n">
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:63)
> at
> org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
> <?xml version="1.0" encoding="UTF-8"?>
> at
> org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:61)
> at
> org.globus.cog.abstraction.coaster.service.job.manager.LocalQueueProcessor.run(LocalQueueProcessor.java:40)
> Caused by:
> org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could
> not submit job (qsub reported an exit code of 1).
> project: ExM<Fault 1001: "The limit of 1024 node hours per user in the
> 'default' queue has been reached\n">
> 
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:113)
> at
> org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:53)
> ... 3 more
> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could
> not submit job (qsub reported an exit code of 1).
> project: ExM<Fault 1001: "The limit of 1024 node hours per user in the
> 'default' queue has been reached\n">
> 
> 
> Is there any ways to tell Swift/Coasters to not do cleanup? If so, is there
> any harm in doing so?
> 
> Thanks,
> Ketan
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user