[Swift-user] sites.xml for cloud
Yadu Nand
yadudoc1729 at gmail.com
Thu Jan 16 11:06:36 CST 2014
Hi Khawar,
Could you copy the /etc/sites/persistent-coasters file in your swift
installation to your $HOME/.swift/sites folder ?
As you noted earlier, the apps file from part04 onwards should reference
"persistent-coasters" as defined in the
sites.xml instead of "cloud".
So, the apps file should look like this :
persistent-coasters sh /bin/bash
And the sites.xml generated should look like this (with some changes for
your settings):
<config>
<pool handle="persistent-coasters">
<execution provider="coaster-persistent"
url="http://localhost:51853"
jobmanager="local:local"/>
<profile namespace="globus" key="workerManager">passive</profile>
<profile namespace="globus" key="jobsPerNode">1</profile>
<profile key="jobThrottle" namespace="karajan">10</profile>
<profile namespace="karajan" key="initialScore">10000</profile>
<filesystem provider="local" url="none" />
<workdirectory>/tmp/swiftwork</workdirectory>
</pool>
</config>
Please let us know how this works out for you.
Thanks,
Yadu
On Wed, Jan 15, 2014 at 2:56 PM, Khawar Ahmad <khawar.ahmad at cern.ch> wrote:
> Hi David,
>
> Yes, I can see this file in swift installed directory.
>
> Regards,
> Khawar
>
> On 15 Jan 2014, at 19:53, "David Kelly" <davidk at ci.uchicago.edu> wrote:
>
> Hi Khawaar,
>
> Sorry that you're running into problems. I think the issue is related to
> message "Cannot find template for persistent-coasters", which is strange
> because that file should be included with Swift. In the place where Swift
> is installed, do you see a file called etc/sites/persistent-coasters?
>
>
> On Wed, Jan 15, 2014 at 12:25 PM, Khawar Ahmad <khawar.ahmad at cern.ch>wrote:
>
>> Hi David,
>>
>> I have done these changes. See the conf file below.
>>
>> xport WORKER_LOCATION=/home/ubuntu
>> export WORKER_HOSTS="192.168.100.3 192.168.100.4"
>> export WORKER_MODE=ssh
>> export WORKER_USERNAME=ubuntu
>> export IPADDR="192.168.100.2"
>> export WORKER_LOG_DIR=/home/ubuntu
>> export WORKER_LOGGING_LEVEL=DEBUG
>> export WORK=/home/ubuntu/work
>> export JOBSPERNODE=1
>> export JOBTHROTTLE=10
>> export SSH_TUNNELING=yes
>>
>> setup.sh went fine without errors. see the output below.
>>
>> Swift version is Swift 0.94.1 swift-r7114 cog-r3803
>> Assuming
>> /home/ubuntu/swift-cloud-tutorial/bin:/home/ubuntu/swift-cloud-tutorial/app:
>> is already at front of PATH
>> Saving /home/ubuntu/.swift/swift.properties in
>> /home/ubuntu/.swift/swift.properties.LTCg
>> Start-coaster-service...
>> Configuration: /home/ubuntu/swift-cloud-tutorial/scs/coaster-service.conf
>> Service address: 192.168.100.2
>> Starting coaster-service
>> Service port: 52871
>> Local port: 60572
>> Generating sites.xml
>> Cannot find template for persistent-coasters
>> Starting worker on 192.168.100.3
>> Starting worker on 192.168.100.4
>>
>>
>> However, the sites.xml file after part03 are empty. see the output
>> below.
>>
>> ubuntu at uwe-vm1:~/swift-cloud-tutorial$ more part0*/sites.xml
>> ::::::::::::::
>> part01/sites.xml
>> ::::::::::::::
>> <?xml version="1.0" encoding="UTF-8"?>
>> <config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
>> <pool handle="localhost">
>> <execution provider="local" />
>> <profile namespace="karajan" key="jobThrottle">0</profile>
>> <profile namespace="karajan" key="initialScore">10000</profile>
>> <filesystem provider="local"/>
>> <workdirectory>.swift/tmp</workdirectory>
>> <profile namespace="swift" key="stagingMethod">local</profile>
>> </pool>
>> </config>
>> ::::::::::::::
>> part02/sites.xml
>> ::::::::::::::
>> <?xml version="1.0" encoding="UTF-8"?>
>> <config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
>> <pool handle="localhost">
>> <execution provider="local" />
>> <profile namespace="karajan" key="jobThrottle">0</profile>
>> <profile namespace="karajan" key="initialScore">10000</profile>
>> <filesystem provider="local"/>
>> <workdirectory>.swift/tmp</workdirectory>
>> <profile namespace="swift" key="stagingMethod">local</profile>
>> </pool>
>> </config>
>> ::::::::::::::
>> part03/sites.xml
>> ::::::::::::::
>> <?xml version="1.0" encoding="UTF-8"?>
>> <config xmlns="http://www.ci.uchicago.edu/swift/SwiftSites">
>> <pool handle="localhost">
>> <execution provider="local" />
>> <profile namespace="karajan" key="jobThrottle">0</profile>
>> <profile namespace="karajan" key="initialScore">10000</profile>
>> <filesystem provider="local"/>
>> <workdirectory>.swift/tmp</workdirectory>
>> <profile namespace="swift" key="stagingMethod">local</profile>
>> </pool>
>> </config>
>> ::::::::::::::
>> part04/sites.xml
>> ::::::::::::::
>> ::::::::::::::
>> part05/sites.xml
>> ::::::::::::::
>> ::::::::::::::
>> part06/sites.xml
>> ::::::::::::::
>>
>>
>> In the tutorial, part4 onwards is described as running on compute nodes
>> and apps file has entry “cloud sh /bin/bash”, however, I could not see how
>> “cloud” would be resolved.
>> This is why, I could not run examples part03 onwards. Any help?
>>
>>
>> Regards,
>> Khawar
>> On 15 Jan 2014, at 16:17, David Kelly <davidk at ci.uchicago.edu> wrote:
>>
>> Hi Khawar,
>>
>> For running with the cloud tutorial, all the settings are defined in a
>> file called scs/coaster-service.conf. The only things you should need to
>> change there is the list of IP addresses, the remote username, and remote
>> directories to use.
>>
>> When you run the command "source setup.sh", it should start the workers
>> on remote sites and create a sites.xml file in each the part directories.
>> Are you getting any errors when you run "source setup.sh"?
>>
>> Thanks,
>> David
>>
>>
>> On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad <khawar.ahmad at cern.ch>wrote:
>>
>>> Hi,
>>>
>>> I am trying to use SWIFT to run sample programs on my test Cloud
>>> setup. I am following a tutorial (
>>> http://swift-lang.org/tutorials/cloud/tutorial.html).
>>> However, the sites.xml file for Part4 is empty and I get the following
>>> error from swift
>>> "Execution failed:
>>> Could not load file sites.xml:
>>> com.thoughtworks.xstream.io.StreamException: : input contained no data"
>>>
>>> Can you tell me what are the settings to run jobs in the Cloud
>>> environment? The Cloud is setup using OpenStack.
>>>
>>> Regards,
>>> Khawar
>>>
>>> _______________________________________________
>>> Swift-user mailing list
>>> Swift-user at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>>
>>
>>
>>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
--
Yadu Nand B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140116/ff62eabc/attachment.html>
More information about the Swift-user
mailing list