[Swift-user] Swift on local resources

[Michael Wilde]

Ah, very cool. Im eager to get more user experience feedback on 
multicore use.

So I will try to hunt down my examples of .ssh configs.

Also, Allan Espinosa used this recently. Allan, can you post details and 
examples?

Thanks!

Mike

On 6/12/09 2:06 PM, Andriy Fedorov wrote:
> Michael,
> 
> Thank you for the advice, I will look into this. This is very helpful.
> I had an impression Lava is not included in the list of schedulers
> supported out of the box, but wanted to check.
> 
> Just a clarification -- I need to access two different types of local
> resources. Cluster (via Lava or Condor) is one, but for the multicore
> nodes we have on the network, which are not part of cluster, the only
> option is to use ssh.
> 
> AF
> 
> 
> 
> On Fri, Jun 12, 2009 at 2:52 PM, Michael Wilde<wilde at mcs.anl.gov> wrote:
>> Andriy,
>>
>> Ben or Mihael may have better ideas, but I offer my thoughts below.
>>
>> On 6/12/09 1:18 PM, Andriy Fedorov wrote:
>>> Hi,
>>>
>>> I am trying to set up Swift with the local cluster and non-cluster
>>> resources in our lab. Here some configuration details.
>>>
>>> Due to technical problems, passphrase login is not possible for the
>>> nodes on local network, and I need to enter password each time.
>>>
>>> For the cluster, I was able to set up passphrase login for the head
>>> node. The cluster is running Lava and Condor schedulers at the same
>>> time, but Lava should be used if possible.
>>>
>>> Two questions:
>>>
>>> (1) is it possible to configure Swift to talk to Lava scheduler?
>> Making Swift talk to a new scheduler means writing a new CoG provider (in
>> Java). You can likely use an existing "data" provider like "local"; you
>> could model the "execution" provider after the "PBS" provider. How hard this
>> is depends on how close Lava is to PBS in nature. (I dont know it). And the
>> provider interface you need to code to is not well documented afaik.
>>
>> I would try the Condor provider. While that provider is less mature and
>> tested than others, it should work, and if it doesnt, we should try to fix
>> it.
>>
>> If possible, make sure a simple condor_submit hello-world works for you
>> first.
>>
>> Run swift on the head/login node; use the "local" data provider.
>>
>> Another route is to use Falkon, but that will be harder and its less
>> supported, so I suggest against this until easier routes are exhausted.
>>
>> I dont think that ssh will get you far, as to leverage the cluster I think
>> you'd need to describe each worker node with a separate sites.xml entry.
>> Thats fine in principle, but a bit awkward, and may have scheduling issues
>> (ie if ssh hangs or dies when you dont own the node).
>>
>> Save ssh as another last resort; I suggest trying Condor first.
>>
>> If needed, people who used ssh recently can send you the info below.
>>
>> - Mike
>>
>>> (2) I am following the instructions on setting up ssh site provider to
>>> use nodes on the local network.
>>>  (2.1) do I need to set up auth.defaults even if I have ssh-agent
>>> running, and can ssh to the remote node without being asked for
>>> password?
>>>  (2.2.) can anybody give me more detailed instructions on how to set
>>> up auth.defaults? I cannot make it work.
>>
>>> Thanks
>>>
>>> Andriy Fedorov
>>> _______________________________________________
>>> Swift-user mailing list
>>> Swift-user at ci.uchicago.edu
>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user