[Swift-devel] using a reservation
Jonathan Monette
jonmon at mcs.anl.gov
Thu Mar 22 13:54:55 CDT 2012
I read this in reading the comments to the JobSoecification source. And using providerAttributed key that Ketan used in his example worked. I will add information to the swiftdevel PBS page regarding this and point to the coaster page for more information.
Thanks Justin.
On Mar 22, 2012, at 13:47, Justin M Wozniak <wozniak at mcs.anl.gov> wrote:
>
> The issue is that Coasters does not know about PBS-specific settings. The providerAttributes profile allows you to pack multiple scheduler-specific settings into an opaque string; Coasters unpacks and passes these through to the underlying scheduler.
>
> Cf. https://sites.google.com/site/swiftdevel/internals/providers/coasters-provider
>
> <profile namespace="globus" key="pbs.resource_list">advres=res_id</profile>
>
> This should work if you use PBS directly.
>
> Justin
>
> On Thu, 22 Mar 2012, Jonathan Monette wrote:
>
>> Using pbs.resource_list in the in the providerAttributes key works. The google site that I was using does not specify this. It seems to lead you to using <profile namespace="globes" key="pbs.resource_list">advres=res_id</profile> instead.
>>
>> On Mar 22, 2012, at 8:03 AM, Michael Wilde wrote:
>>
>>> Thanks, Ketan.
>>>
>>> We should match this against the 0.93 & trunk provider codes:
>>>
>>> pbs.aprun;pbs.mpp;pbs.resource_list=advres=modFTDock.47
>>>
>>> I was wondering if in fact pbs.properties needed to be specified in the manner you show here (above). Im wondering if pbs.resource_list was changed in a revision after you ran this?
>>>
>>> - Mike
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
>>>> To: "Jonathan Monette" <jonmon at mcs.anl.gov>
>>>> Cc: "swift-devel at ci.uchicago.edu Devel" <swift-devel at ci.uchicago.edu>
>>>> Sent: Thursday, March 22, 2012 7:56:38 AM
>>>> Subject: Re: [Swift-devel] using a reservation
>>>> Jon,
>>>>
>>>>
>>>> Here is a sites.xml that I used for a Beagle reservation a while ago
>>>> for modftdock. This worked well on swift-r4252 cog-r3088. See if it
>>>> helps at all comparing yours and this one:
>>>>
>>>> =========
>>>>
>>>>
>>>>
>>>> <config>
>>>> <pool handle="pbs">
>>>> <execution provider="coaster" jobmanager="local:pbs"/>
>>>> <profile namespace="globus" key="project">CI-CCR000013</profile>
>>>> <profile namespace="globus" key="jobsPerNode">24</profile>
>>>> <profile namespace="globus" key="ppn">24</profile>
>>>> <profile namespace="globus" key="maxTime">16000</profile>
>>>> <profile namespace="globus" key="lowOverallocation">100</profile>
>>>> <profile namespace="globus" key="highOverallocation">100</profile>
>>>> <profile namespace="globus" key="providerAttributes">
>>>> pbs.aprun;pbs.mpp;pbs.resource_list=advres=modFTDock.47
>>>> </profile>
>>>> <profile namespace="globus" key="slots">25</profile>
>>>> <profile namespace="globus" key="nodeGranularity">2</profile>
>>>> <profile namespace="globus" key="maxNodes">2</profile>
>>>> <profile namespace="karajan" key="jobThrottle">13.00</profile>
>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>> <filesystem provider="local"/>
>>>> <workdirectory>/lustre/beagle/ketan/labs/modftdock/bgl.reserved.run/swift.workdir</workdirectory>
>>>> </pool>
>>>> </config>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> =========
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Mar 22, 2012 at 12:37 AM, Jonathan Monette <
>>>> jonmon at mcs.anl.gov > wrote:
>>>>
>>>>
>>>> So I have been looking at this. I tried adding my own reservation key
>>>> to PBSExecutor but that does not seem to work. So my question is, does
>>>> this not work because the JobSpecification object does not know to
>>>> look for this attribute? If so, could this be the reason why I cannot
>>>> seem to get the reservation to the PBS script using pbs.properties or
>>>> pbs.resources(I also tried pbs.resource_list as that is what the code
>>>> looks for). Where does the JobSpecification get built? Where is the
>>>> xml sites file parsed? I cannot seem to find this code.
>>>>
>>>>
>>>>
>>>>
>>>> On Mar 21, 2012, at 10:57 PM, Justin M Wozniak wrote:
>>>>
>>>>>
>>>>> I'll take a look at this tomorrow.
>>>>>
>>>>> On Wed, 21 Mar 2012, Michael Wilde wrote:
>>>>>
>>>>>> Jon,
>>>>>>
>>>>>> Regarding the walltime, your sites file mis-spells maxwalltime;
>>>>>> hence the jobs emitted by your script probably dont sum to anything
>>>>>> beyond 17:00h at the 10m default time.
>>>>>>
>>>>>> I dont see why the res isnt making it through to the PBS script.
>>>>>>
>>>>>> - Mike
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Jonathan Monette" < jonmon at mcs.anl.gov >
>>>>>>> To: " swift-devel at ci.uchicago.edu Devel" <
>>>>>>> swift-devel at ci.uchicago.edu >
>>>>>>> Sent: Wednesday, March 21, 2012 6:30:14 PM
>>>>>>> Subject: [Swift-devel] using a reservation
>>>>>>> Hello,
>>>>>>> I am trying to use a reservation I have on Beagle. Here is my
>>>>>>> sites
>>>>>>> file:
>>>>>>>
>>>>>>> <config>
>>>>>>>
>>>>>>>
>>>>>>> <pool handle="local" >
>>>>>>> <execution provider="local" url="none" />
>>>>>>> <profile namespace="karajan" key="jobThrottle">0.5</profile>
>>>>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>>>>> <filesystem provider="local"/>
>>>>>>> <workdirectory>_WORK_/local</workdirectory>
>>>>>>> </pool>
>>>>>>>
>>>>>>>
>>>>>>> <pool handle="beagleRes">
>>>>>>>
>>>>>>>
>>>>>>> <execution provider="coaster" jobmanager="local:pbs"/>
>>>>>>>
>>>>>>>
>>>>>>> <profile namespace="env" key="SWIFT_GEN_SCRIPTS">KEEP</profile>
>>>>>>>
>>>>>>>
>>>>>>> <profile namespace="globus" key="project">CI-MCB000119</profile>
>>>>>>> <profile namespace="globus" key="jobsPerNode">1</profile>
>>>>>>> <profile namespace="globus"
>>>>>>> key="workerLoggingLevel">DEBUG</profile>
>>>>>>> <profile namespace="globus"
>>>>>>> key="workerLoggingDirectory">_WORK_/beagleRes/workers</profile>
>>>>>>> <profile namespace="globus" key="lowOverAllocation">100</profile>
>>>>>>> <profile namespace="globus" key="highOverAllocation">100</profile>
>>>>>>> <profile namespace="globus"
>>>>>>> key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
>>>>>>> <profile namespace="globus" key="maxTime">86400</profile>
>>>>>>> <profile namespace="globus" key="waxWallTime">00:04:00</profile>
>>>>>>> <profile namespace="globus" key="slots">1</profile>
>>>>>>> <profile namespace="globus" key="nodeGranularity">20</profile>
>>>>>>> <profile namespace="globus" key="maxNodes">20</profile>
>>>>>>>
>>>>>>>
>>>>>>> <profile namespace="globus"
>>>>>>> key="pbs.properties">advres=18833.687</profile>
>>>>>>>
>>>>>>>
>>>>>>> <profile namespace="karajan" key="jobThrottle">12.00</profile>
>>>>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>>>>>
>>>>>>>
>>>>>>> <filesystem provider="local"/>
>>>>>>> <workdirectory>_WORK_/beagleRes</workdirectory>
>>>>>>>
>>>>>>>
>>>>>>> </pool>
>>>>>>>
>>>>>>>
>>>>>>> </config>
>>>>>>>
>>>>>>>
>>>>>>> I have tried both pbs.properties and pbs.resources as a sites
>>>>>>> entry, I
>>>>>>> got this information from
>>>>>>> https://sites.google.com/site/swiftdevel/sites/pbs
>>>>>>> However here is the pbs script that has been generated:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> #CoG This script generated by CoG
>>>>>>> #CoG by class: class
>>>>>>> org.globus.cog.abstraction.impl.scheduler.pbs.PBSExecutor
>>>>>>> #CoG on date: 2012/03/21 23:22:35
>>>>>>>
>>>>>>>
>>>>>>> #PBS -S /bin/bash
>>>>>>> #PBS -N Block-0321-2211
>>>>>>> #PBS -m n
>>>>>>> #PBS -A CI-MCB000119
>>>>>>> #PBS -l mppwidth=20,mppnppn=1,mppdepth=24
>>>>>>> #PBS -l walltime=17:00:00
>>>>>>> #PBS -o
>>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.stdout
>>>>>>> #PBS -e
>>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.stderr
>>>>>>> WORKER_LOGGING_LEVEL=DEBUG
>>>>>>> #PBS -v WORKER_LOGGING_LEVEL
>>>>>>> cd / && aprun -n 20 -N 1 -cc none -d 24 -F exclusive /bin/sh -c
>>>>>>> '/usr/bin/perl
>>>>>>> /home/jonmon/.globus/coasters/ cscript9177561070598799820.pl
>>>>>>> http://10.128.2.243:40904 , http://127.0.0.2:40904 ,
>>>>>>> http://192.5.86.103:40904
>>>>>>> 0321-221135-000000
>>>>>>> /lustre/beagle/jonmon/Swift/SciColSim/run163/swiftwork/beagleRes/workers'
>>>>>>> /bin/echo $?
>>>>>>>> /home/jonmon/.globus/scripts/PBS1332885235909759395.submit.exitcode
>>>>>>>
>>>>>>>
>>>>>>> I have asked for a coaster block of 24 hours(my reservation is 96
>>>>>>> hours) but it shows a wall time of 17 hours. Furthermore, the line
>>>>>>> #PBS -l advres=<res_id> is missing so I am not using my
>>>>>>> reservation, I
>>>>>>> just get added to the batch queue and sit there. Does any remember
>>>>>>> how
>>>>>>> to specify a reservation in the sites file for PBS?
>>>>>>> _______________________________________________
>>>>>>> Swift-devel mailing list
>>>>>>> Swift-devel at ci.uchicago.edu
>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Justin M Wozniak
>>>>
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu
>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ketan
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu
>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>
>>> --
>>> Michael Wilde
>>> Computation Institute, University of Chicago
>>> Mathematics and Computer Science Division
>>> Argonne National Laboratory
>>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>
>
> --
> Justin M Wozniak
More information about the Swift-devel
mailing list