[Swift-user] running swift k on multiple instances on EC2, using coaster-service

[Ketan Maheshwari]

Hi Ioan,

I've been extensively using Swift on Amazon EC2 via coasters.

I am using a fedora 8 image.

I use an external host as a submit host so, only perl (for worker.pl; is
installed) and application binaries needs to be on EC2 nodes.

The name of my image is ketan_mars and is on us-east-1 region.

Let me know if you have more questions in this regard.

Regards,
Ketan

On Wed, Oct 24, 2012 at 12:47 PM, Ioan Raicu <iraicu at cs.iit.edu> wrote:

> Hi David,
> Has anyone in the Swift group run Swift on Amazon? Perhaps there are
> already some pre-configured images Iman can use, that already has the
> environment (Linux, Java, Swift, PBS/Condor, GridFTP, NFS/PVFS, etc)
> setup and ready to use. If there are such images, please point Iman in
> the right direction. If not, perhaps after Iman finishes his
> configuration, we can reference his images for others to use on Amazon.
> Ioan
>
> On 10/24/2012 10:28 AM, David Kelly wrote:
> > Iman,
> >
> > There is some documentation on how to use the start-coaster-service
> script at
> http://www.ci.uchicago.edu/swift/guides/release-0.93/siteguide/siteguide.html#_bag_of_workstations.
> You should be able to follow the "bag of workstations" configuration
> example once you have started your EC2 instance. That may help to simplify
> things for you a bit. Please let me know if you have any questions or
> issues with this.
> >
> > Thanks,
> > David
> >
> > ----- Original Message -----
> >> From: "Iman Sadooghi" <isadoogh at iit.edu>
> >> To: swift-user at ci.uchicago.edu
> >> Sent: Tuesday, October 23, 2012 5:46:57 PM
> >> Subject: [Swift-user] running swift k on multiple instances on EC2,
> using coaster-service
> >> Hi everyone
> >>
> >>
> >> I am trying to run a Montage application workflow with swift on
> >> multiple instances of AMAZON EC2.
> >> So far I was able to set up a cluster, and a PVFS files system shared
> >> among the nodes ( using FUSE. so I will have POSIX interface on my
> >> swift work directory ).
> >> I have tried running a simple hello.swift example on multiple nodes
> >> with the coaster. the working directory is the shared folder
> >> (supported by PVFS).
> >> when I run the code using my own tc.data and sites.xml, this will
> >> happen:
> >>
> >>
> >>
> >> (my command) ubuntu at ip-10-244-4-101:~/coaster$ swift -tc.file tc.data
> >> -sites.file sites.xml ~/swift-0.93/examples/swift/tutorial/hello.swift
> >> (results:)
> >> Swift 0.93 swift-r5483 cog-r3339
> >>
> >>
> >> RunID: 20121023-2200-4d3knr72
> >> Progress: time: Tue, 23 Oct 2012 22:00:50 +0000
> >> Find: http://10.244.4.101:1213
> >> Find: keepalive(120), reconnect - http://10.244.4.101:1213
> >> Passive queue processor initialized. Callback URI is
> >> http://10.244.4.101:1212
> >> Progress: time: Tue, 23 Oct 2012 22:01:20 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:01:50 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:02:20 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:02:50 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:03:20 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:03:50 +0000 Submitted:1
> >> Progress: time: Tue, 23 Oct 2012 22:04:20 +0000 Submitted:1
> >>
> >>
> >> and it keeps doing this forever meaning that there is no answer from
> >> worker nodes!
> >> as I checked on worker nodes, the working files are created on the
> >> shared folder, and when i check the running applications, there is a
> >> java application running. but nothing happens.
> >> I have also attached the log file of my hello.swift running in case
> >> you need to take a look at it.
> >> should I consider using pbs, or condor,... I have no idea about how
> >> they work though.
> >>
> >>
> >> I appreciate if anyone can help me with it. Thank you so much.
> >>
> >> Best, --
> >> Iman Sadooghi
> >> Illinois Institute of Technology (IIT)
> >> Data-Intensive Distributed Systems Laboratory
> >>
> >>
> >> _______________________________________________
> >> Swift-user mailing list
> >> Swift-user at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
> --
> =================================================================
> Ioan Raicu, Ph.D.
> Assistant Professor, Illinois Institute of Technology (IIT)
> Guest Research Faculty, Argonne National Laboratory (ANL)
> =================================================================
> Data-Intensive Distributed Systems Laboratory, CS/IIT
> Distributed Systems Laboratory, MCS/ANL
> =================================================================
> Cel:    1-847-722-0876
> Office: 1-312-567-5704
> Email:  iraicu at cs.iit.edu
> Web:    http://www.cs.iit.edu/~iraicu/
> Web:    http://datasys.cs.iit.edu/
> =================================================================
> =================================================================
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>



-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20121024/63f415e1/attachment.html>