[Swift-user] running swift k on multiple instances on EC2, using coaster-service

[Ioan Raicu]

Hi David,
Has anyone in the Swift group run Swift on Amazon? Perhaps there are 
already some pre-configured images Iman can use, that already has the 
environment (Linux, Java, Swift, PBS/Condor, GridFTP, NFS/PVFS, etc) 
setup and ready to use. If there are such images, please point Iman in 
the right direction. If not, perhaps after Iman finishes his 
configuration, we can reference his images for others to use on Amazon.
Ioan

On 10/24/2012 10:28 AM, David Kelly wrote:
> Iman,
>
> There is some documentation on how to use the start-coaster-service script at http://www.ci.uchicago.edu/swift/guides/release-0.93/siteguide/siteguide.html#_bag_of_workstations. You should be able to follow the "bag of workstations" configuration example once you have started your EC2 instance. That may help to simplify things for you a bit. Please let me know if you have any questions or issues with this.
>
> Thanks,
> David
>
> ----- Original Message -----
>> From: "Iman Sadooghi" <isadoogh at iit.edu>
>> To: swift-user at ci.uchicago.edu
>> Sent: Tuesday, October 23, 2012 5:46:57 PM
>> Subject: [Swift-user] running swift k on multiple instances on EC2, using coaster-service
>> Hi everyone
>>
>>
>> I am trying to run a Montage application workflow with swift on
>> multiple instances of AMAZON EC2.
>> So far I was able to set up a cluster, and a PVFS files system shared
>> among the nodes ( using FUSE. so I will have POSIX interface on my
>> swift work directory ).
>> I have tried running a simple hello.swift example on multiple nodes
>> with the coaster. the working directory is the shared folder
>> (supported by PVFS).
>> when I run the code using my own tc.data and sites.xml, this will
>> happen:
>>
>>
>>
>> (my command) ubuntu at ip-10-244-4-101:~/coaster$ swift -tc.file tc.data
>> -sites.file sites.xml ~/swift-0.93/examples/swift/tutorial/hello.swift
>> (results:)
>> Swift 0.93 swift-r5483 cog-r3339
>>
>>
>> RunID: 20121023-2200-4d3knr72
>> Progress: time: Tue, 23 Oct 2012 22:00:50 +0000
>> Find: http://10.244.4.101:1213
>> Find: keepalive(120), reconnect - http://10.244.4.101:1213
>> Passive queue processor initialized. Callback URI is
>> http://10.244.4.101:1212
>> Progress: time: Tue, 23 Oct 2012 22:01:20 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:01:50 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:02:20 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:02:50 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:03:20 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:03:50 +0000 Submitted:1
>> Progress: time: Tue, 23 Oct 2012 22:04:20 +0000 Submitted:1
>>
>>
>> and it keeps doing this forever meaning that there is no answer from
>> worker nodes!
>> as I checked on worker nodes, the working files are created on the
>> shared folder, and when i check the running applications, there is a
>> java application running. but nothing happens.
>> I have also attached the log file of my hello.swift running in case
>> you need to take a look at it.
>> should I consider using pbs, or condor,... I have no idea about how
>> they work though.
>>
>>
>> I appreciate if anyone can help me with it. Thank you so much.
>>
>> Best, --
>> Iman Sadooghi
>> Illinois Institute of Technology (IIT)
>> Data-Intensive Distributed Systems Laboratory
>>
>>
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu at cs.iit.edu
Web:    http://www.cs.iit.edu/~iraicu/
Web:    http://datasys.cs.iit.edu/
=================================================================
=================================================================