[Swift-devel] ssh test case on pads/beagle
Alberto Chavez
alberto_chavez at live.com
Thu Aug 11 08:31:53 CDT 2011
Sure, attached are the output of stdout and stderror, and the log generated by swift.
> Subject: RE: [Swift-devel] ssh test case on pads/beagle
> From: hategan at mcs.anl.gov
> To: alberto_chavez at live.com
> CC: jonmon at mcs.anl.gov; swift-devel at ci.uchicago.edu
> Date: Thu, 11 Aug 2011 00:18:07 -0700
>
> Can you post (a link to) the entire log file? Since it contains both the
> tc.data and sites.xml and the error, it's probably always better to post
> than individual snippets.
>
> On Thu, 2011-08-11 at 01:17 -0500, Alberto Chavez wrote:
> > Sure:
> >
> > <config>
> > <pool handle="ssh">
> > <execution provider="ssh" url="steamroller" jobmanager="ssh"/>
> > <filesystem provider="ssh" url="steamroller" />
> > <profile key="jobThrottle" namespace="karajan">0</profile>
> > <workdirectory>/home/achavez/swiftwork</workdirectory>
> > </pool>
> > </config>
> >
> >
> > ______________________________________________________________________
> > To: alberto_chavez at live.com
> > From: jonmon at mcs.anl.gov
> > CC: hategan at mcs.anl.gov; swift-devel at ci.uchicago.edu
> > Subject: Re: [Swift-devel] ssh test case on pads/beagle
> > Date: Wed, 10 Aug 2011 23:54:24 -0500
> >
> > Could you post the sites file?
> >
> > ----- Reply message -----
> > From: "Alberto Chavez" <alberto_chavez at live.com>
> > Date: Wed, Aug 10, 2011 7:16 pm
> > Subject: [Swift-devel] ssh test case on pads/beagle
> > To: <jonmon at mcs.anl.gov>
> > Cc: "Mihael Hategan" <hategan at mcs.anl.gov>, "Swift Devel"
> > <swift-devel at ci.uchicago.edu>
> >
> >
> >
> > Exit code "127" normally means that a particular function doesn't
> > exist. Are you sure that all those paths to apps exist?
> > > Yes, I doubled check that and those are the right paths to the apps.
> >
> >
> > Also, I am not sure if this is a problem but shouldn't there be a
> > third column in the app file? LIke
> > "ssh echo /bin/echo null null null"
> >
> >
> >
> >
> > Looking at the documentation for the transformation catalog, the
> > structure should be:
> >
> > site, transformation name, executable path, installation status,
> > platform, and profile entries.
> >
> >
> >
> >
> >
> > The installation status and platform fields are not used. Set them
> > to INSTALLED and INTEL32::LINUX respectively.
> >
> > The profiles field should be set to null if no profile entries are to
> > be specified, or should contain the profile entries separated by
> > semicolons.
> >
> >
> > but even when I switch the columns to INSTALLED and INTEL32::LINUX and
> > keep the profiles field set to null, I'm still getting the same exit
> > code.
> >
> >
> > On Aug 10, 2011, at 6:41 PM, Alberto Chavez wrote:
> >
> > I changed my ssh-key, and they worked on the MCS machines
> > because the authorized_keys file has not been updated yet on
> > the CI Machines.
> > I created a new ssh-key using:
> > ssh-keygen -t rsa -b 2048
> > exactly as the MCS site suggested,
> > On the other hand, I still have a problem, I am getting the
> > following error:
> >
> >
> > Swift svn swift-r4978 cog-r3226
> >
> >
> > RunID: 20110810-1819-1cdo2o62
> > Progress: time: Wed, 10 Aug 2011 18:19:42 -0500
> > Exception in cat:
> > Arguments: [data.txt]
> > Host: ssh
> > Directory:
> > 001-catsn-ssh-20110810-1819-1cdo2o62/jobs/9/cat-9jd0g9ek
> > - - -
> > Caused by: null
> > Caused by:
> > org.globus.cog.abstraction.impl.common.execution.JobException:
> > Job failed with an exit code of 127
> > Final status: time: Wed, 10 Aug 2011 18:20:00 -0500
> > Failed:10
> > The following errors have occurred:
> > 1. Job failed with an exit code of 127 (10 times)
> >
> >
> >
> >
> > These are the contents of the log:
> >
> >
> > Execution completed with errors
> >
> >
> > 2011-08-10 18:19:43,251-0500 INFO ConnectionProtocol Freeing
> > channel 0 [Unnamed Channel]
> > 2011-08-10 18:19:43,263-0500 INFO Exec Exit code 127
> > 2011-08-10 18:19:43,269-0500 INFO ConnectionProtocol Freeing
> > channel 0 [Unnamed Channel]
> > 2011-08-10 18:19:43,277-0500 DEBUG vdl:execute2
> > APPLICATION_EXCEPTION jobid=cat-9jd0g9ek - Application
> > exception: null
> > Caused by:
> > org.globus.cog.abstraction.impl.common.execution.JobException:
> > Job failed with an exit code of 127
> > 2011-08-10 18:19:43,280-0500 INFO vdl:execute END_FAILURE
> > thread=0-5-3-1 tr=cat
> > 2011-08-10 18:19:43,281-0500 INFO vdl:execute Exception in
> > cat:
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.fail(FlowNode.java:250)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.fail(FlowNode.java:254)
> > at
> > org.globus.cog.karajan.workflow.nodes.GenerateErrorNode.post(GenerateErrorNode.java:27)
> > at
> > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > at
> > org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
> > at
> > org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29)
> > at
> > org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197)
> > at
> > org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227)
> > at
> > org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:104)
> > at
> > org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40)
> > at java.util.concurrent.Executors
> > $RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask
> > $Sync.innerRun(FutureTask.java:334)
> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at java.util.concurrent.ThreadPoolExecutor
> > $Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:636)
> > 2011-08-10 18:20:00,332-0500 INFO ExecutionContext Detailed
> > exception:
> >
> >
> > Execution completed with errors
> >
> >
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.fail(FlowNode.java:250)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.fail(FlowNode.java:254)
> > at
> > org.globus.cog.karajan.workflow.nodes.GenerateErrorNode.post(GenerateErrorNode.java:27)
> > at
> > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > at
> > org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
> > at
> > org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29)
> > at
> > org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139)
> > at
> > org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197)
> > at
> > org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227)
> > at
> > org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:104)
> > at
> > org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40)
> > at java.util.concurrent.Executors
> > $RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask
> > $Sync.innerRun(FutureTask.java:334)
> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at java.util.concurrent.ThreadPoolExecutor
> > $Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:636)
> >
> > I believe that the problem resides on the TC file because when
> > I run a much simpler SwiftScript like:
> >
> >
> > int i = 9;
> > trace(i);
> >
> >
> > I get the following output:
> >
> >
> > swift traceme.swift -tc.file tc.template.data
> > -sites.file sites.template.xml -config cf
> > Swift svn swift-r4978 cog-r3226
> >
> >
> > RunID: 20110810-1832-buktjj3d
> > Progress: time: Wed, 10 Aug 2011 18:32:30 -0500
> > SwiftScript trace: 9.0
> > Final status: time: Wed, 10 Aug 2011 18:32:30 -0500
> >
> >
> > but as soon as I start using the commands stated the TC file,
> > I get the "exit code 127"
> >
> >
> > My tc file reads:
> >
> >
> > ssh echo /bin/echo null null
> > ssh cat /bin/cat null null
> > ssh ls /bin/ls null null
> > ssh grep /bin/grep null null
> > ssh sort /bin/sort null null
> > ssh paste /bin/paste null null
> > ssh wc /usr/bin/wc null null
> >
> >
> > I am working on the login node of the MCS machine trying to
> > ssh via Swift to steamroller.
> >
> >
> > > Subject: Re: [Swift-devel] ssh test case on pads/beagle
> > > From: hategan at mcs.anl.gov
> > > To: alberto_chavez at live.com
> > > CC: swift-devel at ci.uchicago.edu
> > > Date: Tue, 9 Aug 2011 11:57:06 -0700
> > >
> > > Hmm: Unsupported passphrase algorithm: AES-128-CBC
> > >
> > > I'll try to see how that can be fixed. In the mean time, can
> > you
> > > generate a new key pair with 3DES encryption instead and use
> > that?
> > >
> > > On Tue, 2011-08-09 at 13:43 -0500, Alberto Chavez wrote:
> > > > Hello,
> > > >
> > > >
> > > > I am trying to run a simpler case than ssh-pbs-coaster
> > test case, and
> > > > I'm still having the same error.
> > > > Now I am running only ssh test case
> > > > (/tests/providers/ssh/001-catsn-ssn.swift)
> > > >
> > > >
> > > > The command line is:
> > > > swift -config cf -tc.file tc.template.data -sites.file
> > > > sites.template.xml 001-catsn-ssh.swift
> > > >
> > > >
> > > > The output:
> > > > Swift svn swift-r4861 (swift modified locally) cog-r3183
> > > >
> > > >
> > > > RunID: 20110809-1336-ohte788a
> > > > Progress: time: Tue, 09 Aug 2011 13:36:42 -0500
> > > > Exception in cat:
> > > > Arguments: [data.txt]
> > > > Host: ssh
> > > > Directory:
> > 001-catsn-ssh-20110809-1336-ohte788a/jobs/m/cat-mq74h7ek
> > > > - - -
> > > >
> > > >
> > > > Caused by: null
> > > > Caused by:
> > > >
> > org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: Invalid private key or passphrase
> > > > Caused by:
> > > >
> > com.sshtools.j2ssh.transport.publickey.InvalidSshKeyException:
> > Can't
> > > > read key due to cryptography problems:
> > > > java.security.NoSuchAlgorithmException: Unsupported
> > passphrase
> > > > algorithm: AES-128-CBC
> > > > Progress: time: Tue, 09 Aug 2011 13:36:43 -0500 Selecting
> > site:8
> > > > Submitting:1 Failed:1
> > > > Exception in cat:
> > > > Arguments: [data.txt]
> > > > Host: ssh
> > > > Directory:
> > 001-catsn-ssh-20110809-1336-ohte788a/jobs/n/cat-nq74h7ek
> > > > - - -
> > > >
> > > >
> > > > Caused by: null
> > > > Caused by:
> > > >
> > org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: Invalid private key or passphrase
> > > > Caused by:
> > > >
> > com.sshtools.j2ssh.transport.publickey.InvalidSshKeyException:
> > Can't
> > > > read key due to cryptography problems:
> > > > java.security.NoSuchAlgorithmException: Unsupported
> > passphrase
> > > > algorithm: AES-128-CBC
> > > > Progress: time: Tue, 09 Aug 2011 13:36:44 -0500 Selecting
> > site:7
> > > > Submitting:1 Failed:2
> > > > Exception in cat:
> > > > Arguments: [data.txt]
> > > > Host: ssh
> > > > Directory:
> > 001-catsn-ssh-20110809-1336-ohte788a/jobs/o/cat-oq74h7ek
> > > > - - -
> > > >
> > > >
> > > > Caused by: null
> > > > Caused by:
> > > >
> > org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: Invalid private key or passphrase
> > > > Caused by:
> > > >
> > com.sshtools.j2ssh.transport.publickey.InvalidSshKeyException:
> > Can't
> > > > read key due to cryptography problems:
> > > > java.security.NoSuchAlgorithmException: Unsupported
> > passphrase
> > > > algorithm: AES-128-CBC
> > > > "error_log.log" 105L, 5770C
> > > >
> > > >
> > > > My auth.defaults reads:
> > > >
> > > >
> > > > login1.beagle.ci.uchicago.edu.type=key
> > > > login1.beagle.ci.uchicago.edu.username=achavez
> > > >
> > login1.beagle.ci.uchicago.edu.key=/home/Alberto/.ssh/identity
> > > >
> > > >
> > > > login1.pads.ci.uchicago.edu.type=key
> > > > login1.pads.ci.uchicago.edu.username=achavez
> > > >
> > login1.pads.ci.uchicago.edu.key=/home/Alberto/.ssh/identity
> > > >
> > > >
> > > >
> > > >
> > > > and it has been set to 600, I ommited the passphrase line,
> > but it is
> > > > there, and the passphrase is right because I just verified
> > it in two
> > > > ways:
> > > > 1) by logging to pads and beagle without providing a
> > password
> > > > 2) "changed" the password. I the "new" password is the
> > same as the
> > > > "old" one.
> > > >
> > > > sites.templates.xml:
> > > >
> > > > <config>
> > > > <pool handle="ssh">
> > > > <execution provider="ssh"
> > url="login1.pads.ci.uchicago.edu"
> > > > jobmanager="ssh"/>
> > > > <filesystem provider="ssh"
> > url="login1.pads.ci.uchicago.edu" />
> > > > <profile key="jobThrottle" namespace="karajan">0</profile>
> > > > <workdirectory>/home/achavez/swiftwork</workdirectory>
> > > > </pool>
> > > > </config>
> > > >
> > > >
> > > > config file:
> > > >
> > > > wrapperlog.always.transfer=true
> > > > sitedir.keep=true
> > > > execution.retries=0
> > > > lazy.errors=true
> > > > status.mode=provider
> > > > use.provider.staging=true
> > > > provider.staging.pin.swiftfiles=false
> > > > foreach.max.threads=10
> > > > provenance.log=true
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I also tried a simpler SwiftScript:
> > > >
> > > >
> > > > type filemsg;
> > > >
> > > >
> > > > app (filemsg output) hello(string s)
> > > > {
> > > > echo s stdout=@filename(output);
> > > > }
> > > >
> > > >
> > > > filemsg myfile<"dogcatdinosaur.out">;
> > > > myfile = hello("dog,cat,dinosaur");
> > > >
> > > >
> > > > and I get the following output:
> > > >
> > > >
> > > > Swift svn swift-r4861 (swift modified locally) cog-r3183
> > > >
> > > >
> > > > RunID: 20110809-1343-2es2hel2
> > > > Progress: time: Tue, 09 Aug 2011 13:43:25 -0500
> > > > Exception in echo:
> > > > Arguments: [dog,cat,dinosaur]
> > > > Host: ssh
> > > > Directory:
> > hello_swift-20110809-1343-2es2hel2/jobs/0/echo-0oldh7ek
> > > > - - -
> > > >
> > > >
> > > > Caused by: null
> > > > Caused by:
> > > >
> > org.globus.cog.abstraction.impl.common.task.InvalidSecurityContextException: Invalid private key or passphrase
> > > > Caused by:
> > > >
> > com.sshtools.j2ssh.transport.publickey.InvalidSshKeyException:
> > Can't
> > > > read key due to cryptography problems:
> > > > java.security.NoSuchAlgorithmException: Unsupported
> > passphrase
> > > > algorithm: AES-128-CBC
> > > > Final status: time: Tue, 09 Aug 2011 13:43:26 -0500
> > Failed:1
> > > > The following errors have occurred:
> > > > 1. Can't read key due to cryptography problems:
> > > > java.security.NoSuchAlgorithmException: Unsupported
> > passphrase
> > > > algorithm: AES-128-CBC
> > > >
> > > >
> > > >
> > > >
> > > > any thoughts on this?
> > > > _______________________________________________
> > > > Swift-devel mailing list
> > > > Swift-devel at ci.uchicago.edu
> > >
> > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > >
> > >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110811/15f0b346/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 001-catsn-ssh-20110811-0828-s51oubu6.log
Type: text/x-log
Size: 167486 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110811/15f0b346/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssh-test-output.log
Type: text/x-log
Size: 3559 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110811/15f0b346/attachment-0001.bin>
More information about the Swift-devel
mailing list