[Swift-user] SWIFT Generates "incompatible" PBS scripts

[Mihael Hategan]

Hi,

Yes, I suspect that the problem is that the script, as written
originally by PBSExecutor is not sh-safe, but relies on some specific
version of bash. Out of curiosity, on the machine this doesn't work on,
can you let us know what the versions of /bin/sh and /bin/bash.

Your workaround seems ok, in that I don't think it would cause problems
unless you want to use the PBS provider directly and pass environment
variables to jobs. In any event, we should fix this.

Mihael

On Thu, 2014-04-03 at 06:57 +0800, chen sui wrote:
> Hello SWIFT,
> I found a solution to the problem. It seems to be working for me in this specific situation so far.
> The solution is to comment out lines 242 through 253 in file "cog/modules/provider-localscheduler/src/org/globus/cog/abstraction/impl/scheduler/pbs/PBSExecutor.java" and recompile the JAR file "cog-provider-localscheduler-0.4.jar".
> After this change, PBSExecutor would no longer insert the starred lines and it now works on the cluster I'm using
> Hope this may be useful for people who encounter the same issue.
> Thanks!Tommy Chen
> 
> From: baiypwup at hotmail.com
> To: swift-user at ci.uchicago.edu
> Subject: SWIFT Generates "incompatible" PBS scripts
> Date: Thu, 3 Apr 2014 01:28:39 +0800
> 
> 
> 
> 
> Hello SWIFT,I have been using SWIFT on two different clusters, it has always been running flawlessly on one of the clusters, but it fails on the second one.
> 
> I think I have narrowed down the cause of the problem;
> 
> When I have these lines (the 2 lines with stars in the following example) in a generated PBS script,
> [begin example]
> #PBS -S /bin/bash
> #PBS -N B0402-0012360-0
> #PBS -m n
> (omitted)
> export WORKER_LOGGING_LEVEL=NONE   (*)
> #PBS -v WORKER_LOGGING_LEVEL  (*)
> [end example]
> 
> The last two lines is causing the PBS to be incompatible with one of the clusters I'm using. When I submit such a script, I get the following error.
> 
> [begin Console output]
> [cs900601 at eric2 scripts]$ qsub PBS4684660157119642383.submit
> 506530.eric2
> ** Job deleted, insufficient hours (0 < 96) left in allocation null **
> [end Console output]
> 
> Clearly, for some reason the qsub on this cluster has "forgotten" every "#PBS" directive prior to the "export" line.
> 
> The reason seems to be 
>    1) the cluster I'm using does not support the "-v" switch.
>    2) Does not accept PBS script whose "#PBS" lines have been "interrupted" by an non PBS directive (the "export" above).
> 
> That means, if I am able to make Swift remove the 2 lines during script generation the problem would go away.
> 
> So my question is, is it possible to do this through Swift configurations?
> I have searched the configuration files but not seem to find a way to do it. The generated script says in the 1st line it's generated by a Java class "by class: class org.globus.cog.abstraction.impl.scheduler.pbs.PBSExecutor".
> 
> Is there a way to check out the source of that "PBSExecutor" class to see what is adding these two lines with asterisks in the generated script and disable it?
> 
> Thanks!Tommy Chen
>  		 	   		   		 	   		  
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user