[Swift-user] Swift Coaster Service & Docker Containers

[Michael Wilde]

Matthew,

I would treat this case similar to the configuration you'd use for a set 
of virtual machines.

Start one coaster service for each pool of identical docker containers 
that you want to run.

For each pool of containers, run a Swift worker (worker.pl) in the 
container and have it connect back to the coaster service you designate 
to manage that pool.

I'm assuming that from a docker contained you can connect out to any 
reachable IP address.

Another approach is to treat the containers like a set of ad-hoc compute 
nodes, and ssh into them with automatic coasters using the ssh:local 
jobmanager setting.

We'll try to test and publish a config for such cases. We'd welcome your 
help with that.

- Mike


On 8/29/14, 7:48 PM, Matthew Shaxted wrote:
>
> Hi All,
>
> I'm trying to find a way to run Swift workflows on multi-host docker 
> containers, and wondering if anybody has had success with this.
>
> When I start a docker container and define specific ports to open on 
> the container, they mapped to random ports on the host machine. So for 
> example, I can start a container with an ssh port open from a host 
> "10.1.1.1", and can then access this container across hosts with "ssh 
> compute at 10.1.1.1 --p 49160".
>
> Now I'm hoping to link these docker containers to Swift's 
> start-coaster-service. I think it would be possible and relatively 
> easy if I can say, start the coasters on a series of IP addresses AND 
> ports. So a host's file perhaps would look something like below, and 
> coasters would be started on the correct docker container:
>
> 10.1.1.1: 49160
>
> 10.1.1.2: 34155
>
> 10.1.1.2: 34156 ...
>
> Does this make sense? Is it possible to start coasters by specifying 
> an IP address and port number?
>
> Any thoughts are greatly appreciated.
>
> Thanks,
>
> Matthew
>
> MATTHEW SHAXTED
>
> SKIDMORE, OWINGS & MERRILL LLP
> 224 SOUTH MICHIGAN AVENUE
> CHICAGO, IL 60604
> T  (312) 360-4368
> MATTHEW.SHAXTED at SOM.COM <mailto:MATTHEW.SHAXTED at SOM.COM>
>
> cid:image001.png at 01CF9071.6FB46030 <http://www.som.com/>
>
> The information contained in this communication may be confidential, 
> is intended only for the use of the recipient(s) named above, and may 
> be legally privileged. If the reader of this message is not the 
> intended recipient, you are hereby notified that any dissemination, 
> distribution, or copying of this communication, or any of its 
> contents, is strictly prohibited and may be unlawful. If you have 
> received this communication in error, please return it to the 
> sen­­­der immediately and delete the original message and any copy of 
> it from your computer system. If you have any questions concerning 
> this message, please contact the sender.
>
> http://intranet.som.com/common/admin/file.cfm?f=%2Fresources%2Fcontent%2F5%2F0%2F4%2F4%2F6%2F4%2F0%2F3%2Fdocuments%2Fimagea560bf%2Egif%406e10073b%2E30854c37
>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Mathematics and Computer Science          Computation Institute
Argonne National Laboratory               The University of Chicago

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140829/85d2f186/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 5311 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140829/85d2f186/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 566 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140829/85d2f186/attachment.gif>