[Swift-user] Swift on local resources

[Michael Wilde]

Andriy,

Ben or Mihael may have better ideas, but I offer my thoughts below.

On 6/12/09 1:18 PM, Andriy Fedorov wrote:
> Hi,
> 
> I am trying to set up Swift with the local cluster and non-cluster
> resources in our lab. Here some configuration details.
> 
> Due to technical problems, passphrase login is not possible for the
> nodes on local network, and I need to enter password each time.
> 
> For the cluster, I was able to set up passphrase login for the head
> node. The cluster is running Lava and Condor schedulers at the same
> time, but Lava should be used if possible.
> 
> Two questions:
> 
> (1) is it possible to configure Swift to talk to Lava scheduler?

Making Swift talk to a new scheduler means writing a new CoG provider 
(in Java). You can likely use an existing "data" provider like "local"; 
you could model the "execution" provider after the "PBS" provider. How 
hard this is depends on how close Lava is to PBS in nature. (I dont know 
it). And the provider interface you need to code to is not well 
documented afaik.

I would try the Condor provider. While that provider is less mature and 
tested than others, it should work, and if it doesnt, we should try to 
fix it.

If possible, make sure a simple condor_submit hello-world works for you 
first.

Run swift on the head/login node; use the "local" data provider.

Another route is to use Falkon, but that will be harder and its less 
supported, so I suggest against this until easier routes are exhausted.

I dont think that ssh will get you far, as to leverage the cluster I 
think you'd need to describe each worker node with a separate sites.xml 
entry. Thats fine in principle, but a bit awkward, and may have 
scheduling issues (ie if ssh hangs or dies when you dont own the node).

Save ssh as another last resort; I suggest trying Condor first.

If needed, people who used ssh recently can send you the info below.

- Mike

> (2) I am following the instructions on setting up ssh site provider to
> use nodes on the local network.
>   (2.1) do I need to set up auth.defaults even if I have ssh-agent
> running, and can ssh to the remote node without being asked for
> password?
>   (2.2.) can anybody give me more detailed instructions on how to set
> up auth.defaults? I cannot make it work.


> Thanks
> 
> Andriy Fedorov
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user