[Swift-user] Swift on local resources

[Andriy Fedorov]

Michael,

Thank you for the advice, I will look into this. This is very helpful.
I had an impression Lava is not included in the list of schedulers
supported out of the box, but wanted to check.

Just a clarification -- I need to access two different types of local
resources. Cluster (via Lava or Condor) is one, but for the multicore
nodes we have on the network, which are not part of cluster, the only
option is to use ssh.

AF



On Fri, Jun 12, 2009 at 2:52 PM, Michael Wilde<wilde at mcs.anl.gov> wrote:
> Andriy,
>
> Ben or Mihael may have better ideas, but I offer my thoughts below.
>
> On 6/12/09 1:18 PM, Andriy Fedorov wrote:
>>
>> Hi,
>>
>> I am trying to set up Swift with the local cluster and non-cluster
>> resources in our lab. Here some configuration details.
>>
>> Due to technical problems, passphrase login is not possible for the
>> nodes on local network, and I need to enter password each time.
>>
>> For the cluster, I was able to set up passphrase login for the head
>> node. The cluster is running Lava and Condor schedulers at the same
>> time, but Lava should be used if possible.
>>
>> Two questions:
>>
>> (1) is it possible to configure Swift to talk to Lava scheduler?
>
> Making Swift talk to a new scheduler means writing a new CoG provider (in
> Java). You can likely use an existing "data" provider like "local"; you
> could model the "execution" provider after the "PBS" provider. How hard this
> is depends on how close Lava is to PBS in nature. (I dont know it). And the
> provider interface you need to code to is not well documented afaik.
>
> I would try the Condor provider. While that provider is less mature and
> tested than others, it should work, and if it doesnt, we should try to fix
> it.
>
> If possible, make sure a simple condor_submit hello-world works for you
> first.
>
> Run swift on the head/login node; use the "local" data provider.
>
> Another route is to use Falkon, but that will be harder and its less
> supported, so I suggest against this until easier routes are exhausted.
>
> I dont think that ssh will get you far, as to leverage the cluster I think
> you'd need to describe each worker node with a separate sites.xml entry.
> Thats fine in principle, but a bit awkward, and may have scheduling issues
> (ie if ssh hangs or dies when you dont own the node).
>
> Save ssh as another last resort; I suggest trying Condor first.
>
> If needed, people who used ssh recently can send you the info below.
>
> - Mike
>
>> (2) I am following the instructions on setting up ssh site provider to
>> use nodes on the local network.
>>  (2.1) do I need to set up auth.defaults even if I have ssh-agent
>> running, and can ssh to the remote node without being asked for
>> password?
>>  (2.2.) can anybody give me more detailed instructions on how to set
>> up auth.defaults? I cannot make it work.
>
>
>> Thanks
>>
>> Andriy Fedorov
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
>