[Swift-devel] command line ssh provider...

Thu Jan 12 21:47:16 CST 2012

That is my plan.

On Jan 12, 2012, at 9:45 PM, Michael Wilde wrote:

> I thought that this new ssh-cl provider should not use auth.defaults.
> 
> Jon, I would wait for ideas from Mihael before dismantling your ssh configuration.
> 
> - Mike
> 
> ----- Original Message -----
>> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
>> To: "Jonathan Monette" <jonmon at mcs.anl.gov>
>> Cc: "Michael Wilde" <wilde at mcs.anl.gov>, "Swift Devel" <swift-devel at ci.uchicago.edu>
>> Sent: Thursday, January 12, 2012 9:42:41 PM
>> Subject: Re: [Swift-devel] command line ssh provider...
>> Jon,
>> 
>> 
>> Do you have auth.defaults file in ~/.ssh/ set? It has the following
>> structure:
>> 
>> 
>> 
>> <hostname>.type=key
>> <hostname>.username=<uname>
>> <hostname>.key=/path/to/key
>> <hostname>.passphrase=<passphrase>
>> 
>> 
>> file perm should be 600
>> 
>> Regards,
>> Ketan
>> 
>> 
>> On Thu, Jan 12, 2012 at 9:24 PM, Jonathan Monette < jonmon at mcs.anl.gov
>>> wrote:
>> 
>> 
>> I am getting a different problem. The provider does not seem to be
>> using an agent.
>> 
>> Starting from my macbook I can ssh -A jonmon at login.ci.uchicago.edu and
>> then do ssh -A jonmon at communicado.ci.uchicago.edu and then ssh -A
>> jonmon at bridled.ci.uchicago.edu in the terminal and none of them
>> require a password.
>> 
>> However if I ssh -A jonmon at login.ci.uchicago.edu and then ssh -A
>> jonmon at communicado.ci.uchicago.edu , then start a Swift run that does
>> a simple hostname call on bridled.ci.uchicago.edu I am prompted for my
>> ci password every time.
>> 
>> I am more than certain that this is a configuration issue so I ask for
>> suggestions. My next step is to completely undo all my ssh keys in the
>> authorized key files and start fresh with new keys and passphrases
>> that are not in my macbook keychain. I do not really want to basically
>> revert back to nothing regarding ssh configuration but this seems to
>> be my only alternative. Any suggestions?
>> 
>> 
>> 
>> 
>> On Jan 12, 2012, at 9:19 PM, Michael Wilde wrote:
>> 
>>> The boostrap log shows this:
>>> 
>>> com$ cat ~/coaster-bootstrap-1460623968.log
>>> using plain mode
>>> BS: http://communicado.ci.uchicago.edu:45621
>>> Failed to download bootstrap jar from
>>> http://communicado.ci.uchicago.edu:45621
>>> com$
>>> 
>>> - Mike
>>> 
>>> ----- Original Message -----
>>>> From: "Mihael Hategan" < hategan at mcs.anl.gov >
>>>> To: "Michael Wilde" < wilde at mcs.anl.gov >
>>>> Cc: "Jonathan Monette" < jonmon at mcs.anl.gov >, "Swift Devel" <
>>>> swift-devel at ci.uchicago.edu >
>>>> Sent: Thursday, January 12, 2012 8:34:36 PM
>>>> Subject: Re: [Swift-devel] command line ssh provider...
>>>> Can't test it right now because UCDavis decided to firewall stuff,
>>>> but
>>>> I
>>>> do get the bootstrap script to start and it gets to the wget part.
>>>> 
>>>> So the question is, do you get a bootstrap log?
>>>> 
>>>> On Thu, 2012-01-12 at 13:45 -0600, Michael Wilde wrote:
>>>>> ssh-cl worked for me going from communicado to both login.ci and
>>>>> bridled.
>>>>> 
>>>>> I *assumed* it used my agent because I did not get a password
>>>>> prompt
>>>>> from the swift run. And I dont get a password prompt when running
>>>>> the ssh command line.
>>>>> 
>>>>> It failed when I tried to use coasters with either provider
>>>>> staging
>>>>> (to login.mcs) or localhost/shared workdir (to login.ci ).
>>>>> 
>>>>> The command line and stdout/err for the coaster/local-workdir case
>>>>> is below. The logs are on ci net under ~wilde/swift/lab. Config
>>>>> and
>>>>> sites file was:
>>>>> 
>>>>> com$ cat cf
>>>>> wrapperlog.always.transfer=true
>>>>> sitedir.keep=true
>>>>> execution.retries=0
>>>>> lazy.errors=false
>>>>> status.mode=provider
>>>>> use.provider.staging=false
>>>>> provider.staging.pin.swiftfiles=false
>>>>> 
>>>>> com$ cat sshcl.xml
>>>>> <config>
>>>>> <pool handle="localhost">
>>>>> <execution provider="ssh-cl" url=" login.ci.uchicago.edu "/>
>>>>> <filesystem provider="local"/>
>>>>> <workdirectory>/home/wilde/swiftwork</workdirectory>
>>>>> </pool>
>>>>> </config>
>>>>> com$
>>>>> 
>>>>> com$ cat sshclcoast.xml
>>>>> <config>
>>>>> <pool handle="localhost">
>>>>> <execution provider="coaster" url=" login.ci.uchicago.edu "
>>>>> jobmanager="ssh-cl:local"/>
>>>>> 
>>>>> <profile namespace="globus" key="jobsPerNode">8</profile>
>>>>> <profile namespace="globus" key="slots">1</profile>
>>>>> <profile namespace="globus" key="nodeGranularity">1</profile>
>>>>> <profile namespace="globus" key="maxNodes">1</profile>
>>>>> <profile namespace="karajan" key="jobThrottle">.01</profile>
>>>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>>>> 
>>>>> <filesystem provider="local"/>
>>>>> <workdirectory>/home/wilde/swiftwork</workdirectory>
>>>>> 
>>>>> </pool>
>>>>> </config>
>>>>> com$
>>>>> 
>>>>> 
>>>>> 
>>>>> - Mike
>>>>> 
>>>>> com$ which swift
>>>>> ~/swift/src/trunk/cog/modules/swift/dist/swift-svn/bin/swift
>>>>> com$ pwd
>>>>> /home/wilde/swift/lab
>>>>> com$ swift -tc.file tc -sites.file sshcl.xml -config cf
>>>>> catsn.swift
>>>>> -n=1
>>>>> Swift trunk swift-r5498 cog-r3347
>>>>> 
>>>>> RunID: 20120112-1343-a7mk2zyc
>>>>> Progress: time: Thu, 12 Jan 2012 13:43:04 -0600
>>>>> Final status: Thu, 12 Jan 2012 13:43:04 -0600 Finished
>>>>> successfully:1
>>>>> com$ swift -tc.file tc -sites.file sshclcoast.xml -config cf
>>>>> catsn.swift -n=1
>>>>> Swift trunk swift-r5498 cog-r3347
>>>>> 
>>>>> RunID: 20120112-1343-ql7sn3f7
>>>>> Progress: time: Thu, 12 Jan 2012 13:43:20 -0600
>>>>> Failed to transfer wrapper log for job cat-ihhm6jlk
>>>>> EXCEPTION Exception in cat:
>>>>> Arguments: [data.txt]
>>>>> Host: localhost
>>>>> Directory: catsn-20120112-1343-ql7sn3f7/jobs/i/cat-ihhm6jlk
>>>>> stderr.txt:
>>>>> 
>>>>> stdout.txt:
>>>>> 
>>>>> ----
>>>>> 
>>>>> Caused by: null
>>>>> Caused by:
>>>>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>>>>> Could not submit job
>>>>> Caused by:
>>>>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>>>>> Could not start coaster service
>>>>> Caused by:
>>>>> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
>>>>> Task ended before registration was received.
>>>>> STDOUT: Failed to download bootstrap jar from
>>>>> http://communicado.ci.uchicago.edu:45621
>>>>> 
>>>>> STDERR: This machine accepts SSH public key and One Time Password
>>>>> (OTP) logins only.
>>>>> If you do not have a public key set up, you will be prompted for a
>>>>> password.
>>>>> This is *not* your CI password, but the One Time Password
>>>>> generated
>>>>> from your
>>>>> OTP token. Do not type your CI password, it will not work. If you
>>>>> do
>>>>> not
>>>>> have a token or public key, you will not be able to login.
>>>>> 
>>>>> See http://www.ci.uchicago.edu/faq for more information.
>>>>> 
>>>>> Caused by:
>>>>> org.globus.cog.abstraction.impl.common.execution.JobException: Job
>>>>> failed with an exit code of 1
>>>>> Execution failed:
>>>>> Job failed with an exit code of 1
>>>>> com$
>>>>> 
>>>>> 
>>>>> ----- Original Message -----
>>>>>> From: "Jonathan Monette" < jonmon at mcs.anl.gov >
>>>>>> To: "Mihael Hategan" < hategan at mcs.anl.gov >
>>>>>> Cc: "Swift Devel" < swift-devel at ci.uchicago.edu >, "Michael
>>>>>> Wilde"
>>>>>> < wilde at mcs.anl.gov >
>>>>>> Sent: Thursday, January 12, 2012 1:29:10 PM
>>>>>> Subject: Re: [Swift-devel] command line ssh provider...
>>>>>> Mike,
>>>>>> You mentioned that you were able to use ssh command line provider
>>>>>> using catsn this morning. Was it using agents? Mihael did you
>>>>>> test
>>>>>> using an agent? How do I specify for it to use an agent if
>>>>>> available?
>>>>>> I can do a simple hostname test from communicado to bridled but
>>>>>> it
>>>>>> asks for my password instead of using the agent I have set up.
>>>>>> 
>>>>>> 
>>>>>> On Jan 12, 2012, at 12:21 AM, Mihael Hategan wrote:
>>>>>> 
>>>>>>> ... is in trunk (cog r3347). I was able to start coasters with
>>>>>>> it.
>>>>>>> The
>>>>>>> provider is called "ssh-cl". It is ssh, so ~/.ssh/config and
>>>>>>> agents
>>>>>>> will
>>>>>>> apply. Please test.
>>>>>>> 
>>>>>>> Mihael
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> Swift-devel mailing list
>>>>>>> Swift-devel at ci.uchicago.edu
>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>> 
>>> 
>>> --
>>> Michael Wilde
>>> Computation Institute, University of Chicago
>>> Mathematics and Computer Science Division
>>> Argonne National Laboratory
>>> 
>> 
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>> 
>> 
>> 
>> 
>> --
>> Ketan
> 
> -- 
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
> 