thought i'd revisit this since anjali re-ran this workflow with fewer jobs (~85K) and perhaps the info would be useful. it showed a similar pattern in that it finished all jobs but one (that is, we were missing a single output file) and hung indefinitely on the last 'finished successfully...'<br>
<br>so this discussion seems to have turned mostly to how coasters requests cores. however, i have to say that *generally* in the past when swift/coasters has requested too many cores for the given queue gram complains and you see it in the gram log, which is not the case here. <br>
<br>that said, if you want em: the swift log is in /home/skenny/swift_logs on ci and the coaster log was too big for my home on ci (and has since been appended to so make sure to match the dates with the swift log), but if someone has access to ranger it's in /var/tmp/skenny_swift on login3<br>
<br>we're continuing to use the same swift version and sites file since it's at least helping us push thru much of the work (doing manual resumes/restarts). <br><br>~sk<br><br><div class="gmail_quote">On Fri, Oct 28, 2011 at 11:02 AM, Justin M Wozniak <span dir="ltr"><<a href="mailto:wozniak@mcs.anl.gov">wozniak@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><br>
I think count is the number of processes. PBSExecutor uses it, that may<br>
be a good place to look. In the Coasters context, I think it is the<br>
number of invocations of <a href="http://worker.pl" target="_blank">worker.pl</a> .<br>
<div><div></div><div class="h5"><br>
On Fri, 28 Oct 2011, David Kelly wrote:<br>
<br>
> Just to clarify - when coasters is being used, count represents the<br>
> number of coaster blocks? Then to get the number of cores to request, I<br>
> should use count*workersPerNode?<br>
><br>
> What about in the case where coasters is not used?<br>
><br>
> ----- Original Message -----<br>
>> From: "Mihael Hategan" <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a>><br>
>> To: "David Kelly" <<a href="mailto:davidk@ci.uchicago.edu">davidk@ci.uchicago.edu</a>><br>
>> Cc: "Anjali Raja" <<a href="mailto:anjraja@gmail.com">anjraja@gmail.com</a>>, "Swift Devel" <<a href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>>, "Swift User"<br>
>> <<a href="mailto:swift-user@ci.uchicago.edu">swift-user@ci.uchicago.edu</a>>, "Ketan Maheshwari" <<a href="mailto:ketancmaheshwari@gmail.com">ketancmaheshwari@gmail.com</a>><br>
>> Sent: Thursday, October 20, 2011 9:08:46 PM<br>
>> Subject: Re: [Swift-devel] [Swift-user] gram on ranger<br>
>> On Thu, 2011-10-20 at 21:03 -0500, David Kelly wrote:<br>
>>> Yep, this is using coasters<br>
>>><br>
>><br>
>> Then no. Count is whatever the block allocation algorithm decides it<br>
>> should be.<br>
>><br>
>>>>><br>
>>>>> Should count=32 in the second case? Am I misunderstanding what<br>
>>>>> 'count' is? Is there any way to get the exact number of<br>
>>>>> applications?<br>
>>>><br>
>>>> Coasters?<br>
> _______________________________________________<br>
> Swift-devel mailing list<br>
> <a href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>
> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
><br>
<br>
</div></div><font color="#888888">--<br>
Justin M Wozniak<br>
</font><div><div></div><div class="h5">_______________________________________________<br>
Swift-devel mailing list<br>
<a href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Sarah Kenny<br>Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III<br>University of California Irvine, Dept. of Neurology ~ 773-818-8300<br><br>