<div dir="ltr">Hi Mike,<div><br></div><div>It was a problem with the number of nodes and cores requested via qsub. Once we got GeMTC working with the latest version of Swift in trunk we were able to use the new turbine-aprun-run.zsh which made running on Raven much easier.</div>
<div><br></div><div style>-Scott</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 25, 2013 at 4:13 PM, Michael Wilde <span dir="ltr"><<a href="mailto:wilde@mcs.anl.gov" target="_blank">wilde@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">What was causing the error message "apsched: claim exceeds reservation's memory" ?<br>
<div class="HOEnZb"><div class="h5"><br>
----- Original Message -----<br>
> From: "Justin M Wozniak" <<a href="mailto:wozniak@mcs.anl.gov">wozniak@mcs.anl.gov</a>><br>
> To: <a href="mailto:exm-user@lists.mcs.anl.gov">exm-user@lists.mcs.anl.gov</a><br>
> Sent: Thursday, April 25, 2013 1:19:19 PM<br>
> Subject: Re: [ExM Users] Submitting tasks on Raven<br>
><br>
><br>
> This definitely works now- we were able to launch tasks on 6 GPUs on<br>
> Raven.<br>
><br>
> On 04/25/2013 09:40 AM, Michael Wilde wrote:<br>
> > Can someone help Scott with this?<br>
> ><br>
> > Scott, I think #PBS -m is the email notification flag, not memory.<br>
> ><br>
> > Its complaining that your aprun command is asking for more<br>
> > resources<br>
> > than the qsub command requested for the job.<br>
> ><br>
> > Check that the aprun -n -N -d do not exceed mppwidth etc from pns<br>
> ><br>
> > Test your aprun args with a qsub -I asking for the correct numnber<br>
> > of<br>
> > nodes and cores<br>
> ><br>
> > On 4/24/13, Scott Krieder <<a href="mailto:skrieder@iit.edu">skrieder@iit.edu</a>> wrote:<br>
> >> Hi All,<br>
> >><br>
> >> I'm trying to run noop.tcl with turbine on Raven. I keep getting a<br>
> >> memory<br>
> >> error:<br>
> >> apsched: claim exceeds reservation's memory<br>
> >><br>
> >> I tried a few different values(100, 100M) for<br>
> >> #PBS -m<br>
> >> but I keep getting the error.<br>
> >><br>
> >> Is there a way to let the PBS job take as much memory as it needs?<br>
> >><br>
> >> Thanks,<br>
> >> Scott<br>
> >><br>
> >> =====aprun.sh script that I'm running=====<br>
> >> # USAGE: qsub aprun.sh<br>
> >><br>
> >> # The user should copy and edit the parameters throughout this<br>
> >> script<br>
> >> # marked USER:<br>
> >><br>
> >> # USER: Directory available from compute nodes:<br>
> >> USER_WORK=/ufs/home/users/p01684<br>
> >><br>
> >> # USER: (optional) Change the qstat name<br>
> >> #PBS -N turbine<br>
> >> # USER: Set the job size<br>
> >> #PBS -l mppwidth=1,mppnppn=3,mppdepth=1<br>
> >> # USER: Set the wall time<br>
> >> #PBS -l walltime=10:00<br>
> >> # USER: (optional) Redirect output from its default location<br>
> >> ($PWD)<br>
> >> #PBS -o /ufs/home/users/p01684/pbs.out<br>
> >><br>
> >> #PBS -j oe<br>
> >> #PBS -m n<br>
> >><br>
> >> # USER: Set configuration of Turbine processes<br>
> >> export TURBINE_ENGINES=1<br>
> >> export ADLB_SERVERS=1<br>
> >><br>
> >> echo "Turbine: aprun.sh"<br>
> >> date "+%m/%d/%Y %I:%M%p"<br>
> >> echo<br>
> >><br>
> >> # Be sure we are in an accessible directory<br>
> >> cd $PBS_O_WORKDIR<br>
> >><br>
> >> set -x<br>
> >> # USER: Set Turbine installation path<br>
> >> export TURBINE_HOME=${USER_WORK}/Public/sfw/turbine<br>
> >> # USER: Select program name<br>
> >> # PROGRAM=${USER_WORK}/adlb-data.tcl<br>
> >> PROGRAM=${TURBINE_HOME}/test/noop.tcl<br>
> >><br>
> >> source ${TURBINE_HOME}/scripts/turbine-config.sh<br>
> >> if [[ ${?} != 0 ]]<br>
> >> then<br>
> >> echo "turbine: configuration error!"<br>
> >> exit 1<br>
> >> fi<br>
> >><br>
> >> # Send environment variables to PBS job:<br>
> >> #PBS -v TURBINE_ENGINES ADLB_SERVERS TURBINE_HOME<br>
> >> # USER: Set aprun parameters to agree with PBS -l settings<br>
> >> # aprun -n 1 -N 1 -cc none -d 1 ${TCLSH} ${PROGRAM}<br>
> >> aprun -n 3 -N 3 -cc none -d 1 ${TURBINE_HOME}/bin/turbine<br>
> >> ${PROGRAM}<br>
> >><br>
> >> --<br>
> >> Scott J. Krieder<br>
> >> C: 419-685-0410<br>
> >> E: <a href="mailto:skrieder@iit.edu">skrieder@iit.edu</a><br>
> >> <a href="http://datasys.cs.iit.edu/~skrieder/" target="_blank">http://datasys.cs.iit.edu/~skrieder/</a><br>
> >><br>
><br>
><br>
> --<br>
> Justin M Wozniak<br>
><br>
> _______________________________________________<br>
> ExM-user mailing list<br>
> <a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
><br>
_______________________________________________<br>
ExM-user mailing list<br>
<a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><font face="Times" size="3">Scott J. Krieder<br></font><div><div><font face="Times" size="3">C: 419-685-0410<br></font><div><font face="Times" size="3">E: <a href="mailto:skrieder@iit.edu" target="_blank">skrieder@iit.edu</a><br>
</font><div><font face="Times" size="3"><a href="http://datasys.cs.iit.edu/~skrieder/" target="_blank">http://datasys.cs.iit.edu/~skrieder/</a></font></div></div></div></div>
</div>