[Swift-user] SWIFT Generates "incompatible" PBS scripts

[Mihael Hategan]

I seem to recall that the current solution is the result of "#PBS -v
name=value" having some problems. Does anybody else remember something
of that sort?

Mihael

On Wed, 2014-04-02 at 18:26 -0500, Michael Wilde wrote:
> Here's the ticket I mentioned:
> 
> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1044
> 
> - Mike
> 
> On 4/2/14, 6:25 PM, Michael Wilde wrote:
> > I've also run into the issue of PBS ignoring all directives after the
> > first non-directive line in the file. I think that we either already
> > have a ticket on this or need one; its indeed a Swift bug as far as I
> > can tell.
> >
> > Thanks for reporting it; as Mihael says, we'll create a fix and report
> > back to the swift-user list.
> >
> > - Mike
> >
> > On 4/2/14, 6:21 PM, Mihael Hategan wrote:
> >> Hi,
> >>
> >> Yes, I suspect that the problem is that the script, as written
> >> originally by PBSExecutor is not sh-safe, but relies on some specific
> >> version of bash. Out of curiosity, on the machine this doesn't work on,
> >> can you let us know what the versions of /bin/sh and /bin/bash.
> >>
> >> Your workaround seems ok, in that I don't think it would cause problems
> >> unless you want to use the PBS provider directly and pass environment
> >> variables to jobs. In any event, we should fix this.
> >>
> >> Mihael
> >>
> >> On Thu, 2014-04-03 at 06:57 +0800, chen sui wrote:
> >>> Hello SWIFT,
> >>> I found a solution to the problem. It seems to be working for me in this specific situation so far.
> >>> The solution is to comment out lines 242 through 253 in file "cog/modules/provider-localscheduler/src/org/globus/cog/abstraction/impl/scheduler/pbs/PBSExecutor.java" and recompile the JAR file "cog-provider-localscheduler-0.4.jar".
> >>> After this change, PBSExecutor would no longer insert the starred lines and it now works on the cluster I'm using
> >>> Hope this may be useful for people who encounter the same issue.
> >>> Thanks!Tommy Chen
> >>>
> >>> From: baiypwup at hotmail.com
> >>> To: swift-user at ci.uchicago.edu
> >>> Subject: SWIFT Generates "incompatible" PBS scripts
> >>> Date: Thu, 3 Apr 2014 01:28:39 +0800
> >>>
> >>>
> >>>
> >>>
> >>> Hello SWIFT,I have been using SWIFT on two different clusters, it has always been running flawlessly on one of the clusters, but it fails on the second one.
> >>>
> >>> I think I have narrowed down the cause of the problem;
> >>>
> >>> When I have these lines (the 2 lines with stars in the following example) in a generated PBS script,
> >>> [begin example]
> >>> #PBS -S /bin/bash
> >>> #PBS -N B0402-0012360-0
> >>> #PBS -m n
> >>> (omitted)
> >>> export WORKER_LOGGING_LEVEL=NONE   (*)
> >>> #PBS -v WORKER_LOGGING_LEVEL  (*)
> >>> [end example]
> >>>
> >>> The last two lines is causing the PBS to be incompatible with one of the clusters I'm using. When I submit such a script, I get the following error.
> >>>
> >>> [begin Console output]
> >>> [cs900601 at eric2 scripts]$ qsub PBS4684660157119642383.submit
> >>> 506530.eric2
> >>> ** Job deleted, insufficient hours (0 < 96) left in allocation null **
> >>> [end Console output]
> >>>
> >>> Clearly, for some reason the qsub on this cluster has "forgotten" every "#PBS" directive prior to the "export" line.
> >>>
> >>> The reason seems to be
> >>>      1) the cluster I'm using does not support the "-v" switch.
> >>>      2) Does not accept PBS script whose "#PBS" lines have been "interrupted" by an non PBS directive (the "export" above).
> >>>
> >>> That means, if I am able to make Swift remove the 2 lines during script generation the problem would go away.
> >>>
> >>> So my question is, is it possible to do this through Swift configurations?
> >>> I have searched the configuration files but not seem to find a way to do it. The generated script says in the 1st line it's generated by a Java class "by class: class org.globus.cog.abstraction.impl.scheduler.pbs.PBSExecutor".
> >>>
> >>> Is there a way to check out the source of that "PBSExecutor" class to see what is adding these two lines with asterisks in the generated script and disable it?
> >>>
> >>> Thanks!Tommy Chen
> >>>    		 	   		   		 	   		
> >>> _______________________________________________
> >>> Swift-user mailing list
> >>> Swift-user at ci.uchicago.edu
> >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >> _______________________________________________
> >> Swift-user mailing list
> >> Swift-user at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>