[Swift-devel] using a reservation

Thu Mar 22 13:47:16 CDT 2012

The issue is that Coasters does not know about PBS-specific settings. 
The providerAttributes profile allows you to pack multiple 
scheduler-specific settings into an opaque string; Coasters unpacks and 
passes these through to the underlying scheduler.

Cf. 
https://sites.google.com/site/swiftdevel/internals/providers/coasters-provider

<profile namespace="globus" 
key="pbs.resource_list">advres=res_id</profile>

This should work if you use PBS directly.

 	Justin

On Thu, 22 Mar 2012, Jonathan Monette wrote:

> Using pbs.resource_list in the in the providerAttributes key works. 
> The google site that I was using does not specify this.  It seems to 
> lead you to using <profile namespace="globes" 
> key="pbs.resource_list">advres=res_id</profile> instead.
>
> On Mar 22, 2012, at 8:03 AM, Michael Wilde wrote:
>
>> Thanks, Ketan.
>>
>> We should match this against the 0.93 & trunk provider codes:
>>
>>  pbs.aprun;pbs.mpp;pbs.resource_list=advres=modFTDock.47
>>
>> I was wondering if in fact pbs.properties needed to be specified in the 
>> manner you show here (above).  Im wondering if pbs.resource_list was 
>> changed in a revision after you ran this?
>>
>> - Mike
>>
>>
>> ----- Original Message -----
>>> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
>>> To: "Jonathan Monette" <jonmon at mcs.anl.gov>
>>> Cc: "swift-devel at ci.uchicago.edu Devel" <swift-devel at ci.uchicago.edu>
>>> Sent: Thursday, March 22, 2012 7:56:38 AM
>>> Subject: Re: [Swift-devel] using a reservation
>>> Jon,
>>>
>>>
>>> Here is a sites.xml that I used for a Beagle reservation a while ago
>>> for modftdock. This worked well on swift-r4252 cog-r3088. See if it
>>> helps at all comparing yours and this one:
>>>
>>> =========
>>>
>>>
>>>
>>> <config>
>>> <pool handle="pbs">
>>> <execution provider="coaster" jobmanager="local:pbs"/>
>>> <profile namespace="globus" key="project">CI-CCR000013</profile>
>>> <profile namespace="globus" key="jobsPerNode">24</profile>
>>> <profile namespace="globus" key="ppn">24</profile>
>>> <profile namespace="globus" key="maxTime">16000</profile>
>>> <profile namespace="globus" key="lowOverallocation">100</profile>
>>> <profile namespace="globus" key="highOverallocation">100</profile>
>>> <profile namespace="globus" key="providerAttributes">
>>> pbs.aprun;pbs.mpp;pbs.resource_list=advres=modFTDock.47
>>> </profile>
>>> <profile namespace="globus" key="slots">25</profile>
>>> <profile namespace="globus" key="nodeGranularity">2</profile>
>>> <profile namespace="globus" key="maxNodes">2</profile>
>>> <profile namespace="karajan" key="jobThrottle">13.00</profile>
>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>> <filesystem provider="local"/>
>>> <workdirectory>/lustre/beagle/ketan/labs/modftdock/bgl.reserved.run/swift.workdir</workdirectory>
>>> </pool>
>>> </config>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> =========
>>>
>>>
>>>
>>>
>>> On Thu, Mar 22, 2012 at 12:37 AM, Jonathan Monette <
>>> jonmon at mcs.anl.gov > wrote:
>>>
>>>
>>> So I have been looking at this. I tried adding my own reservation key
>>> to PBSExecutor but that does not seem to work. So my question is, does
>>> this not work because the JobSpecification object does not know to
>>> look for this attribute? If so, could this be the reason why I cannot
>>> seem to get the reservation to the PBS script using pbs.properties or
>>> pbs.resources(I also tried pbs.resource_list as that is what the code
>>> looks for). Where does the JobSpecification get built? Where is the
>>> xml sites file parsed? I cannot seem to find this code.
>>>
>>>
>>>
>>>
>>> On Mar 21, 2012, at 10:57 PM, Justin M Wozniak wrote:
>>>
>>>>
>>>> I'll take a look at this tomorrow.
>>>>
>>>> On Wed, 21 Mar 2012, Michael Wilde wrote:
>>>>
>>>>> Jon,
>>>>>
>>>>> Regarding the walltime, your sites file mis-spells maxwalltime;
>>>>> hence the jobs emitted by your script probably dont sum to anything
>>>>> beyond 17:00h at the 10m default time.
>>>>>
>>>>> I dont see why the res isnt making it through to the PBS script.
>>>>>
>>>>> - Mike
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Jonathan Monette" < jonmon at mcs.anl.gov >
>>>>>> To: " swift-devel at ci.uchicago.edu Devel" <
>>>>>> swift-devel at ci.uchicago.edu >
>>>>>> Sent: Wednesday, March 21, 2012 6:30:14 PM
>>>>>> Subject: [Swift-devel] using a reservation
>>>>>> Hello,
>>>>>> I am trying to use a reservation I have on Beagle. Here is my
>>>>>> sites
>>>>>> file:
>>>>>>
>>>>>> <config>
>>>>>>
>>>>>>
>>>>>> <pool handle="local" >
>>>>>> <execution provider="local" url="none" />
>>>>>> <profile namespace="karajan" key="jobThrottle">0.5</profile>
>>>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>>>> <filesystem provider="local"/>
>>>>>> <workdirectory>_WORK_/local</workdirectory>
>>>>>> </pool>
>>>>>>
>>>>>>
>>>>>> <pool handle="beagleRes">
>>>>>>
>>>>>>
>>>>>> <execution provider="coaster" jobmanager="local:pbs"/>
>>>>>>
>>>>>>
>>>>>> <profile namespace="env" key="SWIFT_GEN_SCRIPTS">KEEP</profile>
>>>>>>
>>>>>>
>>>>>> <profile namespace="globus" key="project">CI-MCB000119</profile>
>>>>>> <profile namespace="globus" key="jobsPerNode">1</profile>
>>>>>> <profile namespace="globus"
>>>>>> key="workerLoggingLevel">DEBUG</profile>
>>>>>> <profile namespace="globus"
>>>>>> key="workerLoggingDirectory">_WORK_/beagleRes/workers</profile>
>>>>>> <profile namespace="globus" key="lowOverAllocation">100</profile>
>>>>>> <profile namespace="globus" key="highOverAllocation">100</profile>
>>>>>> <profile namespace="globus"
>>>>>> key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
>>>>>> <profile namespace="globus" key="maxTime">86400</profile>
>>>>>> <profile namespace="globus" key="waxWallTime">00:04:00</profile>
>>>>>> <profile namespace="globus" key="slots">1</profile>
>>>>>> <profile namespace="globus" key="nodeGranularity">20</profile>
>>>>>> <profile namespace="globus" key="maxNodes">20</profile>
>>>>>>
>>>>>>
>>>>>> <profile namespace="globus"
>>>>>> key="pbs.properties">advres=18833.687</profile>
>>>>>>
>>>>>>
>>>>>> <profile namespace="karajan" key="jobThrottle">12.00</profile>
>>>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>>>>
>>>>>>
>>>>>> <filesystem provider="local"/>
>>>>>> <workdirectory>_WORK_/beagleRes</workdirectory>
>>>>>>
>>>>>>
>>>>>> </pool>
>>>>>>
>>>>>>
>>>>>> </config>
>>>>>>
>>>>>>
>>>>>> I have tried both pbs.properties and pbs.resources as a sites
>>>>>> entry, I
>>>>>> got this information from
>>>>>> https://sites.google.com/site/swiftdevel/sites/pbs
>>>>>> However here is the pbs script that has been generated:
>>>>>>
>>>>>>
>>>>>>
>>>>>> #CoG This script generated by CoG
>>>>>> #CoG by class: class
>>>>>> org.globus.cog.abstraction.impl.scheduler.pbs.PBSExecutor
>>>>>> #CoG on date: 2012/03/21 23:22:35
>>>>>>
>>>>>>
>>>>>> #PBS -S /bin/bash
>>>>>> #PBS -N Block-0321-2211
>>>>>> #PBS -m n
>>>>>> #PBS -A CI-MCB000119
>>>>>> #PBS -l mppwidth=20,mppnppn=1,mppdepth=24
>>>>>> #PBS -l walltime=17:00:00
>>>>>> #PBS -o
>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.stdout
>>>>>> #PBS -e
>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.stderr
>>>>>> WORKER_LOGGING_LEVEL=DEBUG
>>>>>> #PBS -v WORKER_LOGGING_LEVEL
>>>>>> cd / && aprun -n 20 -N 1 -cc none -d 24 -F exclusive /bin/sh -c
>>>>>> '/usr/bin/perl
>>>>>> /home/jonmon/.globus/coasters/ cscript9177561070598799820.pl
>>>>>> http://10.128.2.243:40904 , http://127.0.0.2:40904 ,
>>>>>> http://192.5.86.103:40904
>>>>>> 0321-221135-000000
>>>>>> /lustre/beagle/jonmon/Swift/SciColSim/run163/swiftwork/beagleRes/workers'
>>>>>> /bin/echo $?
>>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.exitcode
>>>>>>
>>>>>>
>>>>>> I have asked for a coaster block of 24 hours(my reservation is 96
>>>>>> hours) but it shows a wall time of 17 hours. Furthermore, the line
>>>>>> #PBS -l advres=<res_id> is missing so I am not using my
>>>>>> reservation, I
>>>>>> just get added to the batch queue and sit there. Does any remember
>>>>>> how
>>>>>> to specify a reservation in the sites file for PBS?
>>>>>> _______________________________________________
>>>>>> Swift-devel mailing list
>>>>>> Swift-devel at ci.uchicago.edu
>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>>
>>>>>
>>>>
>>>> --
>>>> Justin M Wozniak
>>>
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>
>>>
>>>
>>>
>>> --
>>> Ketan
>>>
>>>
>>>
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>
>> --
>> Michael Wilde
>> Computation Institute, University of Chicago
>> Mathematics and Computer Science Division
>> Argonne National Laboratory
>>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>

-- 
Justin M Wozniak