[Swift-user] multisite passive workers

[Allan Espinosa]

I was using multiply pool entries because each site has a different
gridftp url.

But with provider staging I no longer need that?

-Allan

2010/10/4 Mihael Hategan <hategan at mcs.anl.gov>:
> On Mon, 2010-10-04 at 16:10 -0500, Allan Espinosa wrote:
>> Ahh...
>>
>> I'm just getting one URL service .  Is it because I have the same url
>> for each site?
>
> Pretty much. Why do you want to have different sites? Just start one
> coaster service on localhost and connect workers from wherever.
>
>>
>> With this, for each entry, should I specify a port manually like
>> url="https://communicado.ci.uchicago.edu:1984" ,
>> url="https://communicado.ci.uchicago.edu:1985" ... ?
>>
>> swift-r3649 cog-r2890
>>
>> RunID: osg
>> Progress:
>> Progress:  Initializing:1  Selecting site:817
>> Progress:  Selecting site:1273  Initializing site shared directory:27
>> Stage in:1
>> Passive queue processor initialized. Callback URI is http://128.135.125.17:50003
>> Progress:  Selecting site:1100  Initializing site shared directory:24
>> Stage in:9  Submitted:168
>> Progress:  Selecting site:1058  Initializing site shared directory:23
>> Stage in:4  Submitted:216
>> Progress:  Selecting site:1058  Initializing site shared directory:23
>> Stage in:4  Submitted:216
>> Progress:  Selecting site:1041  Initializing site shared directory:23
>> Stage in:21  Submitted:216
>>
>>
>> site entries;
>>   <pool handle="UJ-OSG">
>>     <execution provider="coaster" url="communicado.ci.uchicago.edu"
>>         jobmanager="local:local" />
>>
>>     <profile namespace="globus" key="workerManager">passive</profile>
>>
>>     <profile namespace="karajan" key="initialScore">20.0</profile>
>>     <profile namespace="karajan" key="jobThrottle">2.7</profile>
>>
>>     <profile namespace="globus" key="lowOverallocation">36</profile>
>>
>>     <gridftp  url="gsiftp://osg-ce.grid.uj.ac.za"/>
>>     <workdirectory>/nfs/data/osg_store/engage-scec/swift_scratch</workdirectory>
>>   </pool>
>>
>>   <pool handle="UMissHEP">
>>     <execution provider="coaster" url="communicado.ci.uchicago.edu"
>>         jobmanager="local:local" />
>>
>>     <profile namespace="globus" key="workerManager">passive</profile>
>>
>>     <profile namespace="karajan" key="initialScore">20.0</profile>
>>     <profile namespace="karajan" key="jobThrottle">0.44</profile>
>>
>>     <profile namespace="globus" key="lowOverallocation">36</profile>
>>
>>     <gridftp  url="gsiftp://umiss001.hep.olemiss.edu"/>
>>     <workdirectory>/osgremote/osg_data/engage-scec/swift_scratch</workdirectory>
>>   </pool>
>>
>>   <pool handle="USCMS-FNAL-WC1">
>>     <execution provider="coaster" url="communicado.ci.uchicago.edu"
>>         jobmanager="local:local" />
>>
>>     <profile namespace="globus" key="workerManager">passive</profile>
>>
>>     <profile namespace="karajan" key="initialScore">20.0</profile>
>>     <profile namespace="karajan" key="jobThrottle">68.7</profile>
>>
>>     <profile namespace="globus" key="lowOverallocation">36</profile>
>>
>>     <gridftp  url="gsiftp://cmsosgce3.fnal.gov"/>
>>     <workdirectory>/uscms_grid/data/engage-scec/swift_scratch</workdirectory>
>>   </pool>
>>
>>   <pool handle="UTA_SWT2">
>>     <execution provider="coaster" url="communicado.ci.uchicago.edu"
>>         jobmanager="local:local" />
>>
>>     <profile namespace="globus" key="workerManager">passive</profile>
>>
>>     <profile namespace="karajan" key="initialScore">20.0</profile>
>>     <profile namespace="karajan" key="jobThrottle">9.46</profile>
>>
>>     <profile namespace="globus" key="lowOverallocation">36</profile>
>>
>>     <gridftp  url="gsiftp://gk04.swt2.uta.edu"/>
>>     <workdirectory>/cluster/grid/data/engage-scec/swift_scratch</workdirectory>
>>   </pool>
>>
>>   <pool handle="WQCG-Harvard-OSG">
>>     <execution provider="coaster" url="communicado.ci.uchicago.edu"
>>         jobmanager="local:local" />
>>
>>     <profile namespace="globus" key="workerManager">passive</profile>
>>
>>     <profile namespace="karajan" key="initialScore">20.0</profile>
>>     <profile namespace="karajan" key="jobThrottle">0.22</profile>
>>
>>     <profile namespace="globus" key="lowOverallocation">36</profile>
>>
>>     <gridftp  url="gsiftp://tuscany.med.harvard.edu"/>
>>     <workdirectory>/osg/storage/data/engage-scec/swift_scratch</workdirectory>
>>   </pool>
>>
>>
>> 2010/10/4 Mihael Hategan <hategan at mcs.anl.gov>:
>> > On Mon, 2010-10-04 at 13:21 -0500, Allan Espinosa wrote:
>> >> Hi,
>> >>
>> >> I noticed that there is only one callback URI when I have multiple
>> >> sites with a passive worker configuration.  How does the coaster
>> >> service know which site the registering worker belong to?
>> >
>> > You should get one URL for each service, unless there is some failure in
>> > starting a service.
>> >
>> >>
>> >> My test jobs maybe failing because the site swift initializes for a
>> >> particular worker maybe the wrong one.
>> >>
>> >> thanks,
>> >> -Allan