<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi,
<div><br>
</div>
<div>Further to the message below, I received this from our cluster administrator:</div>
<div><br>
</div>
<div>There isn't a parallel environment called 1way on Legion. Ian spent some time modifying the source code for Swift 0.94.1 to get it to submit jobs on Legion, but then found that those modifications no longer work for the later versions.<br>
<br>
Here's what he had to do for the earlier version:<br>
<br>
Modify the code in swift-0.94/cog/modules/provider-localscheduler/src/org/globus/cog/abstraction/impl/scheduler/sge/SGEExecutor.java:<br>
<br>
* find the function private void verifyQueueInformation() and delete the contents, replacing with just return;<br>
* delete the line: writeAttr("queue", "-q ", wr);<br>
<br>
and then, because it expects to be able to inspect the queue settings itself to determine cores per node and jobs per node, I ham-fistedly changed those to be hard-coded to 1 in the same file, changing:<br>
<br>
String queue = (String)spec.getAttribute("queue");<br>
<br>
int coresPerNode = Integer.valueOf(getAttribute(spec, "coresPerNode",<br>
String.valueOf(poller.getQueueInformation(queue).getSlots())));<br>
int jobsPerNode = Integer.valueOf(getAttribute(spec, "jobsPerNode",<br>
String.valueOf(coresPerNode)));<br>
int coresToRequest = ( count * jobsPerNode + coresPerNode - 1) / coresPerNode * coresPerNode;<br>
<br>
to<br>
<br>
//String queue = (String)spec.getAttribute("queue");<br>
<br>
int coresPerNode = 1;<br>
int jobsPerNode = 1;<br>
int coresToRequest = 1;<br>
<br>
In cog/modules/provider-localscheduler/src/org/globus/cog/abstraction/impl/scheduler/sge/SGEExecutor.java:<br>
<br>
change:<br>
<br>
writeWallTime(wr);<br>
writeSoftWallTime(wr);<br>
<br>
if (spec.getStdInput() != null) {<br>
wr.write("#$ -i " + quote(spec.getStdInput()) + '\n');<br>
}<br>
wr.write("#$ -o " + quote(stdout) + '\n');<br>
wr.write("#$ -e " + quote(stderr) + '\n');<br>
<br>
if (!spec.getEnvironmentVariableNames().isEmpty()) {<br>
<br>
to:<br>
<br>
writeWallTime(wr);<br>
writeSoftWallTime(wr);<br>
<br>
if (spec.getStdInput() != null) {<br>
wr.write("#$ -i " + quote(spec.getStdInput()) + '\n');<br>
}<br>
wr.write("#$ -o " + quote(stdout) + '\n');<br>
wr.write("#$ -e " + quote(stderr) + '\n');<br>
wr.write("#$ -jsv /shared/ucl/apps/sge_support/clean_variables_from_jobenv.jsv\n");<br>
<br>
if (!spec.getEnvironmentVariableNames().isEmpty()) {<br>
<br>
As the above suggests, I've put that JSV script in /shared/ucl/apps/sge_support in case it's needed for anything else. The bug which makes it useful has been fixed in the version of SoGE we're using after the upgrade, so it shouldn't be necessary for too long.
(That part is to do with stripping out variables containing % characters).<br>
<div>
<div>On May 1, 2015, at 4:27 PM, Altaweel, Mark <<a href="mailto:tcrnma3@live.ucl.ac.uk">tcrnma3@live.ucl.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi,
<div><br>
</div>
<div>I am trying to use Swift (<font face="Menlo"><span style="font-size: 11px;">swift-0.96-sge-mod) and trying to run a script on SGE for the local cluster. Is there any clear reason for the error (below).</span></font></div>
<div><span style="font-family: Menlo; font-size: 11px;"><br>
</span></div>
<div><span style="font-family: Menlo; font-size: 11px;">My swift.conf setup is:</span></div>
<div><span style="font-family: Menlo; font-size: 11px;"><br>
</span></div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">site.Legion {</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> execution {</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> type: "coaster"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> jobManager: "local:sge"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> URL : "localhost"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> options {</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> maxJobs: 2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> nodeGranularity: 1</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> maxNodesPerJob: 2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> tasksPerNode: 1</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> jobProject: "AllUsers"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> jobQueue: "Tarvek"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> maxJobTime: "1800"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> }</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> }</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> maxParallelTasks : 3</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> initialParallelTasks : 2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> staging: local</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> workDirectory: "/tmp/"${env.USER}</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> app.ALL {</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> executable: "*"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> maxWallTime: "00:05:00"</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> }</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">}</div>
</div>
<div><span style="font-family: Menlo; font-size: 11px;"><br>
</span></div>
<div><span style="font-family: Menlo; font-size: 11px;"><br>
</span></div>
<div><font face="Menlo"><span style="font-size: 11px;">I get the following error:</span></font></div>
<div><font face="Menlo"><span style="font-size: 11px;"><br>
</span></font></div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">RunID: run009</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Warning: The @ syntax for function invocation is deprecated</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Warning: Variable spans, defined on line 52, might have multiple conflicting writers</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Progress: Fri, 01 May 2015 16:19:29+0100</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Number of parameter combinations: 2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Stride: 1</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Begin: 1, End: 1</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Begin: 2, End: 2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Progress: Fri, 01 May 2015 16:19:30+0100 Submitted:2</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Error: No parallel environment specified</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Could not submit job (qsub reported an exit code of 1). </div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Unable to run job: job rejected: the requested parallel environment "1way" does not exist.Exiting.</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Execution failed:</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Exception in sh:</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> Arguments: [repast_instance.sh, /imports/home1/tcrnma3/Scratch/UrbanModel/, 1, 1, 1, urf_2.txt]</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> Host: Legion</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"> Directory: repast-run009/jobs/9/sh-9xg6568m</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>exception @ swift-int-staging.k, line: 174</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Caused by: </div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>exception @ swift-int-staging.k, line: 170</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Caused by: Block task failed: Error submitting block task</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could not submit job (qsub reported an exit code of 1). </div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Unable to run job: job rejected: the requested parallel environment "1way" does not exist.Exiting.</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:62)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.common.task.ExecutionTaskHandler.submit(ExecutionTaskHandler.java:61)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.coaster.service.job.manager.BlockTaskSubmitter.run(BlockTaskSubmitter.java:70)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Caused by: org.globus.cog.abstraction.impl.scheduler.common.ProcessException: Could not submit job (qsub reported an exit code of 1). </div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Unable to run job: job rejected: the requested parallel environment "1way" does not exist.Exiting.</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.scheduler.common.AbstractExecutor.start(AbstractExecutor.java:116)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.scheduler.sge.SGEExecutor.start(SGEExecutor.java:192)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>at org.globus.cog.abstraction.impl.scheduler.common.AbstractJobSubmissionTaskHandler.submit(AbstractJobSubmissionTaskHandler.java:52)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span>... 3 more</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo; min-height: 13px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><span class="Apple-tab-span" style="white-space:pre"></span></div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>