[Swift-user] Swift on local resources

[Mihael Hategan]

On Fri, 2009-06-12 at 16:17 -0400, Andriy Fedorov wrote:
> Allan,
> 
> Thank you for the example.
> 
> I have exactly same setup, but it doesn't work for me. I suspect the
> reason is that I am unable to set up my environment to work with
> passphrase. I can only log in with password. I wonder if there is any
> workaround...

If you want to use password authentication, remove the key and
passphrase lines and add <host>.password=xxx.

However, in some cases being asked by ssh for a password may mean
"keyboard interactive authentication" which is not the same as
"username/password authentication" and may not work with the ssh
provider. Do an ssh -v and you'll see the list of authentication methods
that the server allows.

> 
> AF
> 
> 
> 
> On Fri, Jun 12, 2009 at 4:04 PM, Allan
> Espinosa<aespinosa at cs.uchicago.edu> wrote:
> > Hi Andriy and Mike,
> >
> > here is my example ~/.ssh/auth.defaults for executing jobs on
> > tp-login1.ci.uchicago.edu:
> >
> > [aespinosa at tp-login2 ~]$ cat .ssh/auth.defaults
> > tp-login1.ci.uchicago.edu.type=key
> > tp-login1.ci.uchicago.edu.username=aespinosa
> > tp-login1.ci.uchicago.edu.key=/home/aespinosa/.ssh/id_dsa
> > tp-login1.ci.uchicago.edu.passphrase=XXXXXXXX
> >
> > We have used falkon and coasters before for multi-core configurations.
> >  but for a single multi-core machine, i believe you can get away with
> > having multiple entries of the same host in the sites.xml file using
> > the ssh-provider.
> >
> > ie:
> >
> > <config>
> > <pool handle="CORE0">
> >   <execution provider="ssh"... />
> >   ...
> > </pool>
> > <pool handle="CORE1">
> >   ...
> > </pool>
> > </config>
> >
> > 2009/6/12 Michael Wilde <wilde at mcs.anl.gov>:
> >> Ah, very cool. Im eager to get more user experience feedback on multicore
> >> use.
> >>
> >> So I will try to hunt down my examples of .ssh configs.
> >>
> >> Also, Allan Espinosa used this recently. Allan, can you post details and
> >> examples?
> >>
> >> Thanks!
> >>
> >> Mike
> >>
> >> On 6/12/09 2:06 PM, Andriy Fedorov wrote:
> >>>
> >>> Michael,
> >>>
> >>> Thank you for the advice, I will look into this. This is very helpful.
> >>> I had an impression Lava is not included in the list of schedulers
> >>> supported out of the box, but wanted to check.
> >>>
> >>> Just a clarification -- I need to access two different types of local
> >>> resources. Cluster (via Lava or Condor) is one, but for the multicore
> >>> nodes we have on the network, which are not part of cluster, the only
> >>> option is to use ssh.
> >>>
> >>> AF
> >>>
> >>>
> >>>
> >>> On Fri, Jun 12, 2009 at 2:52 PM, Michael Wilde<wilde at mcs.anl.gov> wrote:
> >>>>
> >>>> Andriy,
> >>>>
> >>>> Ben or Mihael may have better ideas, but I offer my thoughts below.
> >>>>
> >>>> On 6/12/09 1:18 PM, Andriy Fedorov wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I am trying to set up Swift with the local cluster and non-cluster
> >>>>> resources in our lab. Here some configuration details.
> >>>>>
> >>>>> Due to technical problems, passphrase login is not possible for the
> >>>>> nodes on local network, and I need to enter password each time.
> >>>>>
> >>>>> For the cluster, I was able to set up passphrase login for the head
> >>>>> node. The cluster is running Lava and Condor schedulers at the same
> >>>>> time, but Lava should be used if possible.
> >>>>>
> >>>>> Two questions:
> >>>>>
> >>>>> (1) is it possible to configure Swift to talk to Lava scheduler?
> >>>>
> >>>> Making Swift talk to a new scheduler means writing a new CoG provider (in
> >>>> Java). You can likely use an existing "data" provider like "local"; you
> >>>> could model the "execution" provider after the "PBS" provider. How hard
> >>>> this
> >>>> is depends on how close Lava is to PBS in nature. (I dont know it). And
> >>>> the
> >>>> provider interface you need to code to is not well documented afaik.
> >>>>
> >>>> I would try the Condor provider. While that provider is less mature and
> >>>> tested than others, it should work, and if it doesnt, we should try to
> >>>> fix
> >>>> it.
> >>>>
> >>>> If possible, make sure a simple condor_submit hello-world works for you
> >>>> first.
> >>>>
> >>>> Run swift on the head/login node; use the "local" data provider.
> >>>>
> >>>> Another route is to use Falkon, but that will be harder and its less
> >>>> supported, so I suggest against this until easier routes are exhausted.
> >>>>
> >>>> I dont think that ssh will get you far, as to leverage the cluster I
> >>>> think
> >>>> you'd need to describe each worker node with a separate sites.xml entry.
> >>>> Thats fine in principle, but a bit awkward, and may have scheduling
> >>>> issues
> >>>> (ie if ssh hangs or dies when you dont own the node).
> >>>>
> >>>> Save ssh as another last resort; I suggest trying Condor first.
> >>>>
> >>>> If needed, people who used ssh recently can send you the info below.
> >>>>
> >>>> - Mike
> >>>>
> >>>>> (2) I am following the instructions on setting up ssh site provider to
> >>>>> use nodes on the local network.
> >>>>>  (2.1) do I need to set up auth.defaults even if I have ssh-agent
> >>>>> running, and can ssh to the remote node without being asked for
> >>>>> password?
> >>>>>  (2.2.) can anybody give me more detailed instructions on how to set
> >>>>> up auth.defaults? I cannot make it work.
> >>>>
> >>>>> Thanks
> >>>>>
> >>>>> Andriy Fedorov
> >>>>> _______________________________________________
> >
> >
> >
> > --
> > Allan M. Espinosa <http://allan.88-mph.net/blog>
> > PhD student, Computer Science
> > University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
> >
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user