[Swift-devel] command line ssh provider...

Michael Wilde wilde at mcs.anl.gov
Thu Jan 12 21:45:48 CST 2012 

I thought that this new ssh-cl provider should not use auth.defaults.

Jon, I would wait for ideas from Mihael before dismantling your ssh configuration.

- Mike

----- Original Message -----
> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> To: "Jonathan Monette" <jonmon at mcs.anl.gov>
> Cc: "Michael Wilde" <wilde at mcs.anl.gov>, "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, January 12, 2012 9:42:41 PM
> Subject: Re: [Swift-devel] command line ssh provider...
> Jon,
> 
> 
> Do you have auth.defaults file in ~/.ssh/ set? It has the following
> structure:
> 
> 
> 
> <hostname>.type=key
> <hostname>.username=<uname>
> <hostname>.key=/path/to/key
> <hostname>.passphrase=<passphrase>
> 
> 
> file perm should be 600
> 
> Regards,
> Ketan
> 
> 
> On Thu, Jan 12, 2012 at 9:24 PM, Jonathan Monette < jonmon at mcs.anl.gov
> > wrote:
> 
> 
> I am getting a different problem. The provider does not seem to be
> using an agent.
> 
> Starting from my macbook I can ssh -A jonmon at login.ci.uchicago.edu and
> then do ssh -A jonmon at communicado.ci.uchicago.edu and then ssh -A
> jonmon at bridled.ci.uchicago.edu in the terminal and none of them
> require a password.
> 
> However if I ssh -A jonmon at login.ci.uchicago.edu and then ssh -A
> jonmon at communicado.ci.uchicago.edu , then start a Swift run that does
> a simple hostname call on bridled.ci.uchicago.edu I am prompted for my
> ci password every time.
> 
> I am more than certain that this is a configuration issue so I ask for
> suggestions. My next step is to completely undo all my ssh keys in the
> authorized key files and start fresh with new keys and passphrases
> that are not in my macbook keychain. I do not really want to basically
> revert back to nothing regarding ssh configuration but this seems to
> be my only alternative. Any suggestions?
> 
> 
> 
> 
> On Jan 12, 2012, at 9:19 PM, Michael Wilde wrote:
> 
> > The boostrap log shows this:
> >
> > com$ cat ~/coaster-bootstrap-1460623968.log
> > using plain mode
> > BS: http://communicado.ci.uchicago.edu:45621
> > Failed to download bootstrap jar from
> > http://communicado.ci.uchicago.edu:45621
> > com$
> >
> > - Mike
> >
> > ----- Original Message -----
> >> From: "Mihael Hategan" < hategan at mcs.anl.gov >
> >> To: "Michael Wilde" < wilde at mcs.anl.gov >
> >> Cc: "Jonathan Monette" < jonmon at mcs.anl.gov >, "Swift Devel" <
> >> swift-devel at ci.uchicago.edu >
> >> Sent: Thursday, January 12, 2012 8:34:36 PM
> >> Subject: Re: [Swift-devel] command line ssh provider...
> >> Can't test it right now because UCDavis decided to firewall stuff,
> >> but
> >> I
> >> do get the bootstrap script to start and it gets to the wget part.
> >>
> >> So the question is, do you get a bootstrap log?
> >>
> >> On Thu, 2012-01-12 at 13:45 -0600, Michael Wilde wrote:
> >>> ssh-cl worked for me going from communicado to both login.ci and
> >>> bridled.
> >>>
> >>> I *assumed* it used my agent because I did not get a password
> >>> prompt
> >>> from the swift run. And I dont get a password prompt when running
> >>> the ssh command line.
> >>>
> >>> It failed when I tried to use coasters with either provider
> >>> staging
> >>> (to login.mcs) or localhost/shared workdir (to login.ci ).
> >>>
> >>> The command line and stdout/err for the coaster/local-workdir case
> >>> is below. The logs are on ci net under ~wilde/swift/lab. Config
> >>> and
> >>> sites file was:
> >>>
> >>> com$ cat cf
> >>> wrapperlog.always.transfer=true
> >>> sitedir.keep=true
> >>> execution.retries=0
> >>> lazy.errors=false
> >>> status.mode=provider
> >>> use.provider.staging=false
> >>> provider.staging.pin.swiftfiles=false
> >>>
> >>> com$ cat sshcl.xml
> >>> <config>
> >>> <pool handle="localhost">
> >>> <execution provider="ssh-cl" url=" login.ci.uchicago.edu "/>
> >>> <filesystem provider="local"/>
> >>> <workdirectory>/home/wilde/swiftwork</workdirectory>
> >>> </pool>
> >>> </config>
> >>> com$
> >>>
> >>> com$ cat sshclcoast.xml
> >>> <config>
> >>> <pool handle="localhost">
> >>> <execution provider="coaster" url=" login.ci.uchicago.edu "
> >>> jobmanager="ssh-cl:local"/>
> >>>
> >>> <profile namespace="globus" key="jobsPerNode">8</profile>
> >>> <profile namespace="globus" key="slots">1</profile>
> >>> <profile namespace="globus" key="nodeGranularity">1</profile>
> >>> <profile namespace="globus" key="maxNodes">1</profile>
> >>> <profile namespace="karajan" key="jobThrottle">.01</profile>
> >>> <profile namespace="karajan" key="initialScore">10000</profile>
> >>>
> >>> <filesystem provider="local"/>
> >>> <workdirectory>/home/wilde/swiftwork</workdirectory>
> >>>
> >>> </pool>
> >>> </config>
> >>> com$
> >>>
> >>>
> >>>
> >>> - Mike
> >>>
> >>> com$ which swift
> >>> ~/swift/src/trunk/cog/modules/swift/dist/swift-svn/bin/swift
> >>> com$ pwd
> >>> /home/wilde/swift/lab
> >>> com$ swift -tc.file tc -sites.file sshcl.xml -config cf
> >>> catsn.swift
> >>> -n=1
> >>> Swift trunk swift-r5498 cog-r3347
> >>>
> >>> RunID: 20120112-1343-a7mk2zyc
> >>> Progress: time: Thu, 12 Jan 2012 13:43:04 -0600
> >>> Final status: Thu, 12 Jan 2012 13:43:04 -0600 Finished
> >>> successfully:1
> >>> com$ swift -tc.file tc -sites.file sshclcoast.xml -config cf
> >>> catsn.swift -n=1
> >>> Swift trunk swift-r5498 cog-r3347
> >>>
> >>> RunID: 20120112-1343-ql7sn3f7
> >>> Progress: time: Thu, 12 Jan 2012 13:43:20 -0600
> >>> Failed to transfer wrapper log for job cat-ihhm6jlk
> >>> EXCEPTION Exception in cat:
> >>> Arguments: [data.txt]
> >>> Host: localhost
> >>> Directory: catsn-20120112-1343-ql7sn3f7/jobs/i/cat-ihhm6jlk
> >>> stderr.txt:
> >>>
> >>> stdout.txt:
> >>>
> >>> ----
> >>>
> >>> Caused by: null
> >>> Caused by:
> >>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> >>> Could not submit job
> >>> Caused by:
> >>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> >>> Could not start coaster service
> >>> Caused by:
> >>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> >>> Task ended before registration was received.
> >>> STDOUT: Failed to download bootstrap jar from
> >>> http://communicado.ci.uchicago.edu:45621
> >>>
> >>> STDERR: This machine accepts SSH public key and One Time Password
> >>> (OTP) logins only.
> >>> If you do not have a public key set up, you will be prompted for a
> >>> password.
> >>> This is *not* your CI password, but the One Time Password
> >>> generated
> >>> from your
> >>> OTP token. Do not type your CI password, it will not work. If you
> >>> do
> >>> not
> >>> have a token or public key, you will not be able to login.
> >>>
> >>> See http://www.ci.uchicago.edu/faq for more information.
> >>>
> >>> Caused by:
> >>> org.globus.cog.abstraction.impl.common.execution.JobException: Job
> >>> failed with an exit code of 1
> >>> Execution failed:
> >>> Job failed with an exit code of 1
> >>> com$
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Jonathan Monette" < jonmon at mcs.anl.gov >
> >>>> To: "Mihael Hategan" < hategan at mcs.anl.gov >
> >>>> Cc: "Swift Devel" < swift-devel at ci.uchicago.edu >, "Michael
> >>>> Wilde"
> >>>> < wilde at mcs.anl.gov >
> >>>> Sent: Thursday, January 12, 2012 1:29:10 PM
> >>>> Subject: Re: [Swift-devel] command line ssh provider...
> >>>> Mike,
> >>>> You mentioned that you were able to use ssh command line provider
> >>>> using catsn this morning. Was it using agents? Mihael did you
> >>>> test
> >>>> using an agent? How do I specify for it to use an agent if
> >>>> available?
> >>>> I can do a simple hostname test from communicado to bridled but
> >>>> it
> >>>> asks for my password instead of using the agent I have set up.
> >>>>
> >>>>
> >>>> On Jan 12, 2012, at 12:21 AM, Mihael Hategan wrote:
> >>>>
> >>>>> ... is in trunk (cog r3347). I was able to start coasters with
> >>>>> it.
> >>>>> The
> >>>>> provider is called "ssh-cl". It is ssh, so ~/.ssh/config and
> >>>>> agents
> >>>>> will
> >>>>> apply. Please test.
> >>>>>
> >>>>> Mihael
> >>>>>
> >>>>> _______________________________________________
> >>>>> Swift-devel mailing list
> >>>>> Swift-devel at ci.uchicago.edu
> >>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >>>
> >
> > --
> > Michael Wilde
> > Computation Institute, University of Chicago
> > Mathematics and Computer Science Division
> > Argonne National Laboratory
> >
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> --
> Ketan

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory

More information about the Swift-devel mailing list