<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: Times New Roman; font-size: 12pt; color: #000000'>David, Jon - see also this thread for possible pointers:<div><br></div><div>http://lists.ci.uchicago.edu/pipermail/swift-devel/2011-July/008541.html<div><br>- Mike</div><div><br><hr id="zwchr"><blockquote style="border-left:2px solid rgb(16, 16, 255);margin-left:5px;padding-left:5px;"><b>From: </b>"Jonathan Monette" <jonmon@mcs.anl.gov><br><b>To: </b>"David Kelly" <davidk@ci.uchicago.edu><br><b>Cc: </b>"swift-devel Devel" <swift-devel@ci.uchicago.edu><br><b>Sent: </b>Friday, August 26, 2011 8:02:30 PM<br><b>Subject: </b>Re: [Swift-devel] Notes from 0.93 meeting<br><br>I don't think it matters but it was a thought. When I was working with swift and globus online I had to set it to an Il address and not the dns name. But that might have been for a different reason.<br><br>----- Reply message -----<br>From: "David Kelly" <davidk@ci.uchicago.edu><br>Date: Fri, Aug 26, 2011 7:13 pm<br>Subject: [Swift-devel] Notes from 0.93 meeting<br>To: "Jonathan Monette" <jonmon@mcs.anl.gov><br>Cc: "Mihael Hategan" <hategan@mcs.anl.gov>, "swift-devel Devel" <swift-devel@ci.uchicago.edu><br><br><br>I set it to communicado.ci.uchicago.edu. I'll try again with IP address.<br><br>----- Original Message -----<br>> From: "Jonathan Monette" <jonmon@mcs.anl.gov><br>> To: "David Kelly" <davidk@ci.uchicago.edu><br>> Cc: "Mihael Hategan" <hategan@mcs.anl.gov>, "swift-devel Devel" <swift-devel@ci.uchicago.edu><br>> Sent: Friday, August 26, 2011 6:54:29 PM<br>> Subject: Re: [Swift-devel] Notes from 0.93 meeting<br>> Did you set GLOBUS_HOSTNAME to communicado.ci.uchicago.edu or probably<br>> better the ip-address of communicado?<br>> On Aug 26, 2011, at 6:52 PM, David Kelly wrote:<br>> <br>> > I tried setting GLOBUS_HOSTNAME on communicado. The gram log file is<br>> > no longer created, but I still don't see any jobs being submitted?<br>> ><br>> > There is a new set of logs at<br>> > www.ci.uchicago.edu/~davidk/ranger-gt2-logs2.tar.gz<br>> ><br>> > David<br>> ><br>> > ----- Original Message -----<br>> >> From: "Mihael Hategan" <hategan@mcs.anl.gov><br>> >> To: "David Kelly" <davidk@ci.uchicago.edu><br>> >> Cc: "swift-devel Devel" <swift-devel@ci.uchicago.edu>, "Jonathan<br>> >> Monette" <jonmon@mcs.anl.gov><br>> >> Sent: Friday, August 26, 2011 1:42:13 PM<br>> >> Subject: Re: [Swift-devel] Notes from 0.93 meeting<br>> >> "The job manager failed to open stderr" tends to happen when you<br>> >> have<br>> >> GLOBUS_HOSTNAME set incorrectly.<br>> >><br>> >> On Fri, 2011-08-26 at 13:38 -0500, David Kelly wrote:<br>> >>> When I am trying to run the script now, Swift does not seem to be<br>> >>> submitting the jobs correctly. Nothing it showing up in qstat.<br>> >>><br>> >>> I noticed that a gram log gets created in my home directory that<br>> >>> says:<br>> >>> ts=2011-08-26T17:30:03.910618Z id=27215 event=gram.job.end<br>> >>> level=ERROR gramid=/16145868447994515851/17606392074284884670/<br>> >>> job_status=4 status=-73 reason="the job manager failed to open<br>> >>> stdout"<br>> >>><br>> >>> I'm guessing this is the cause of the problem. Bugs #153 and #215<br>> >>> were related to similar problems with stdout and gt2/sge.<br>> >>><br>> >>> The full logs are at<br>> >>> <a href="http://www.ci.uchicago.edu/~davidk/ranger-gt2-logs.tar.gz" target="_blank">http://www.ci.uchicago.edu/~davidk/ranger-gt2-logs.tar.gz</a><br>> >>><br>> >>> Thanks,<br>> >>> David<br>> >>><br>> >>><br>> >>> ----- Original Message -----<br>> >>>> From: "Mihael Hategan" <hategan@mcs.anl.gov><br>> >>>> To: "Jonathan Monette" <jonmon@mcs.anl.gov><br>> >>>> Cc: "swift-devel Devel" <swift-devel@ci.uchicago.edu><br>> >>>> Sent: Thursday, August 25, 2011 5:31:34 PM<br>> >>>> Subject: Re: [Swift-devel] Notes from 0.93 meeting<br>> >>>> On Thu, 2011-08-25 at 17:18 -0500, Jonathan Monette wrote:<br>> >>>>> I can send mail to ci support and cc mike to it and ask what<br>> >>>>> they<br>> >>>>> can<br>> >>>>> do.<br>> >>>>><br>> >>>>> Mihael, is there anyway for Swift to give a little more feedback<br>> >>>>> besides unknown CA or is that a jglobus problem?<br>> >>>><br>> >>>> It's a jglobus problem.<br>> >>>><br>> >>>> That in itself may not be a big issue, but jglobus is now being<br>> >>>> heavily<br>> >>>> re-organized by the globus team, so I'm not sure what the best<br>> >>>> long-term<br>> >>>> strategy is here.<br>> >>>>><br>> >>>>> ----- Reply message -----<br>> >>>>> From: "Sarah Kenny" <skenny@uchicago.edu><br>> >>>>> Date: Thu, Aug 25, 2011 5:11 pm<br>> >>>>> Subject: [Swift-devel] Notes from 0.93 meeting<br>> >>>>> To: "Jonathan Monette" <jonmon@mcs.anl.gov><br>> >>>>> Cc: "Mihael Hategan" <hategan@mcs.anl.gov>, "swift-devel Devel"<br>> >>>>> <swift-devel@ci.uchicago.edu><br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>> if i had a nickel for every time i dealt with this i'd be rich!<br>> >>>>> :)<br>> >>>>> actually, now that i'm looking at our uci machines i actually<br>> >>>>> have<br>> >>>>> them updating hourly...so, maybe you want to ask the admins to<br>> >>>>> do<br>> >>>>> that<br>> >>>>> to avoid a full day of confusion whenever they expire :P<br>> >>>>><br>> >>>>> *usually* i can't gsissh either if the certs have expired but,<br>> >>>>> yeah,<br>> >>>>> they must be using different CA's now for that on ranger as<br>> >>>>> mihael<br>> >>>>> suggests...<br>> >>>>><br>> >>>>> On Thu, Aug 25, 2011 at 2:46 PM, Jonathan Monette<br>> >>>>> <jonmon@mcs.anl.gov><br>> >>>>> wrote:<br>> >>>>> True. I did not think that each mechanism would use<br>> >>>>> different<br>> >>>>> CAs. We might want to ask ci support to update the grid<br>> >>>>> certs<br>> >>>>> more frequently then to avoid this situation.<br>> >>>>><br>> >>>>><br>> >>>>> On Aug 25, 2011, at 4:42 PM, Mihael Hategan wrote:<br>> >>>>><br>> >>>>>> On Thu, 2011-08-25 at 16:40 -0500, Jonathan Monette<br>> >>>>>> wrote:<br>> >>>>>>> That is weird. If you were able to gsissh to ranger I<br>> >>>>> would assume<br>> >>>>>>> that you are able to globus-url-copy to ranger.<br>> >>>>>><br>> >>>>>> Not if the two use different CAs. Or if a password was<br>> >>>>>> typed<br>> >>>>> at the ssh<br>> >>>>>> login.<br>> >>>>>><br>> >>>>>>> Anyways, what Sarah said should work. I would assume<br>> >>>>>>> that<br>> >>>>> ci would<br>> >>>>>>> update more frequently to avoid this problem.<br>> >>>>>>> On Aug 25, 2011, at 4:38 PM, Sarah Kenny wrote:<br>> >>>>>>><br>> >>>>>>>> communicado's certs<br>> >>>>>>>> (/etc/grid-security/certificates)<br>> >>>>>>>> are<br>> >>>>>>>> out-of-date...if you copy<br>> >>>>> ranger's /etc/grid-security/certificates<br>> >>>>>>>> directory to communicado and point yr X509_CERT_DIR<br>> >>>>>>>> to<br>> >>>>>>>> it<br>> >>>>> you can<br>> >>>>>>>> get a job thru (a simple globus-job-run with my<br>> >>>>>>>> vaild<br>> >>>>>>>> cert<br>> >>>>> fails<br>> >>>>>>>> from communicado at the moment if i don't do this).<br>> >>>>>>>><br>> >>>>>>>> i set our machines at uci to update daily...i think<br>> >>>>>>>> it's<br>> >>>>> less<br>> >>>>>>>> frequently at ci...<br>> >>>>>>>><br>> >>>>>>>> On Thu, Aug 25, 2011 at 2:17 PM, Mihael Hategan<br>> >>>>>>>> <hategan@mcs.anl.gov> wrote:<br>> >>>>>>>> Can you try a globus-url-copy to<br>> >>>>>>>> gridftp.ranger?<br>> >>>>>>>><br>> >>>>>>>> gridftp.ranger seems to have the NCSA myproxy<br>> >>>>>>>> CA.<br>> >>>>> You say<br>> >>>>>>>> you have the<br>> >>>>>>>> proper certificates dir in your<br>> >>>>>>>> X509_CERT_DIR,<br>> >>>>>>>> and<br>> >>>>> that<br>> >>>>>>>> directory<br>> >>>>>>>> contains the TACC root cert. So it should<br>> >>>>>>>> work.<br>> >>>>>>>> And<br>> >>>>> so<br>> >>>>>>>> should swift.<br>> >>>>>>>><br>> >>>>>>>> Though I think that jglobus should be more<br>> >>>>>>>> clear<br>> >>>>> about<br>> >>>>>>>> "Unknown ca"<br>> >>>>>>>> errors. At least the name of the unknown CA<br>> >>>>>>>> should<br>> >>>>> be part<br>> >>>>>>>> of the error<br>> >>>>>>>> message.<br>> >>>>>>>><br>> >>>>>>>><br>> >>>>>>>> On Thu, 2011-08-25 at 15:55 -0500, David<br>> >>>>>>>> Kelly<br>> >>>>> wrote:<br>> >>>>>>>>> $ grid-proxy-info -all<br>> >>>>>>>>> subject : /C=US/O=National Center for<br>> >>>>>>>>> Supercomputing<br>> >>>>>>>> Applications/CN=David Kelly<br>> >>>>>>>>> issuer : /C=US/O=National Center for Supercomputing<br>> >>>>>>>> Applications/OU=Certificate<br>> >>>>>>>> Authorities/CN=MyProxy<br>> >>>>>>>>> identity : /C=US/O=National Center for<br>> >>>>>>>>> Supercomputing<br>> >>>>>>>> Applications/CN=David Kelly<br>> >>>>>>>>> type : end entity credential<br>> >>>>>>>>> strength : 1024 bits<br>> >>>>>>>>> path : /tmp/x509up_u1878<br>> >>>>>>>>> timeleft : 9:56:53<br>> >>>>>>>>><br>> >>>>>>>>><br>> >>>>>>>>> ----- Original Message -----<br>> >>>>>>>>>> From: "Mihael Hategan" <hategan@mcs.anl.gov><br>> >>>>>>>>>> To: "David Kelly" <davidk@ci.uchicago.edu><br>> >>>>>>>>>> Cc: "Ketan Maheshwari"<br>> >>>>>>>>>> <ketancmaheshwari@gmail.com>,<br>> >>>>>>>> "swift-devel Devel"<br>> >>>>>>>> <swift-devel@ci.uchicago.edu><br>> >>>>>>>>>> Sent: Thursday, August 25, 2011 3:42:57 PM<br>> >>>>>>>>>> Subject: Re: [Swift-devel] Notes from 0.93 meeting<br>> >>>>>>>>>> Odd. Can you paste the output of 'grid-proxy-info<br>> >>>>>>>>>> -all'?<br>> >>>>>>>>>><br>> >>>>>>>>>> On Thu, 2011-08-25 at 15:18 -0500, David Kelly<br>> >>>>>>>>>> wrote:<br>> >>>>>>>>>>> Sure, here is the full log:<br>> >>>>>>>>>>><br>> >>>>>>>>>>><br>> >>>>>>>><br>> >>>>> <a href="http://www.ci.uchicago.edu/~davidk/001-catsn-ranger-20110825-1515-5tydro91.log" target="_blank">http://www.ci.uchicago.edu/~davidk/001-catsn-ranger-20110825-1515-5tydro91.log</a><br>> >>>>>>>>>>><br>> >>>>>>>>>>> ----- Original Message -----<br>> >>>>>>>>>>>> From: "Mihael Hategan" <hategan@mcs.anl.gov><br>> >>>>>>>>>>>> To: "David Kelly" <davidk@ci.uchicago.edu><br>> >>>>>>>>>>>> Cc: "Ketan Maheshwari"<br>> >>>>>>>>>>>> <ketancmaheshwari@gmail.com>,<br>> >>>>>>>> "swift-devel<br>> >>>>>>>>>>>> Devel" <swift-devel@ci.uchicago.edu><br>> >>>>>>>>>>>> Sent: Thursday, August 25, 2011 2:43:31 PM<br>> >>>>>>>>>>>> Subject: Re: [Swift-devel] Notes from 0.93<br>> >>>>>>>>>>>> meeting<br>> >>>>>>>>>>>> It's possible that the CA dir on Ranger is not<br>> >>>>>>>> properly set up.<br>> >>>>>>>>>>>> Can<br>> >>>>>>>>>>>> you<br>> >>>>>>>>>>>> post the full log?<br>> >>>>>>>>>>>><br>> >>>>>>>>>>>> On Thu, 2011-08-25 at 13:56 -0500, David Kelly<br>> >>>>>>>> wrote:<br>> >>>>>>>>>>>>> Those environment variables were not set up. I<br>> >>>>>>>> have them defined<br>> >>>>>>>>>>>>> now, but I'm still getting the same error.<br>> >>>>>>>>>>>>><br>> >>>>>>>>>>>>> [davidk@communicado ranger]$ env |grep 509<br>> >>>>>>>>>>>>> X509_CERT_DIR=/opt/osg-1.2.16/globus/TRUSTED_CA<br>> >>>>>>>>>>>>> X509_CADIR=/opt/osg-1.2.16/globus/TRUSTED_CA<br>> >>>>>>>>>>>>><br>> >>>>>>>>>>>>> [davidk@communicado ranger]$ swift -sites.file<br>> >>>>>>>> sites.xml<br>> >>>>>>>>>>>>> -tc.file<br>> >>>>>>>>>>>>> tc.data 001-catsn-ranger.swift<br>> >>>>>>>>>>>>> Swift svn swift-r4987 (swift modified locally)<br>> >>>>>>>> cog-r3229<br>> >>>>>>>>>>>>><br>> >>>>>>>>>>>>> RunID: 20110825-1352-f1v940b4<br>> >>>>>>>>>>>>> Progress: time: Thu, 25 Aug 2011 13:52:59 -0500<br>> >>>>>>>>>>>>> Progress: time: Thu, 25 Aug 2011 13:53:00 -0500<br>> >>>>>>>> Selecting site:7<br>> >>>>>>>>>>>>> Initializing site shared directory:3<br>> >>>>>>>>>>>>> Execution failed:<br>> >>>>>>>>>>>>> Authentication failed [Caused by: Failure<br>> >>>>>>>> unspecified at<br>> >>>>>>>>>>>>> GSS-API<br>> >>>>>>>>>>>>> level [Caused by: Unknown CA]]<br>> >>>>>>>>>>>>><br>> >>>>>>>>>>>>><br>> >>>>>>>>>>>>> ----- Original Message -----<br>> >>>>>>>>>>>>>> From: "Ketan Maheshwari"<br>> >>>>>>>> <ketancmaheshwari@gmail.com><br>> >>>>>>>>>>>>>> To: "David Kelly" <davidk@ci.uchicago.edu><br>> >>>>>>>>>>>>>> Cc: "Jonathan Monette" <jonmon@mcs.anl.gov>,<br>> >>>>>>>> "swift-devel<br>> >>>>>>>>>>>>>> Devel"<br>> >>>>>>>>>>>>>> <swift-devel@ci.uchicago.edu><br>> >>>>>>>>>>>>>> Sent: Thursday, August 25, 2011 1:32:50 PM<br>> >>>>>>>>>>>>>> Subject: Re: [Swift-devel] Notes from 0.93<br>> >>>>>>>> meeting<br>> >>>>>>>>>>>>>> Hi,<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> Are your CADIR and CACERT env vars set up?<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> [communicado:swiftgrid]$ echo $X509_CADIR<br>> >>>>>>>>>>>>>> /opt/osg-1.2.16/globus/TRUSTED_CA<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> [communicado:swiftgrid]$ echo $X509_CERT_DIR<br>> >>>>>>>>>>>>>> /opt/osg-1.2.16/globus/TRUSTED_CA<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> On Thu, Aug 25, 2011 at 1:29 PM, David Kelly <<br>> >>>>>>>>>>>>>> davidk@ci.uchicago.edu<br>> >>>>>>>>>>>>>>> wrote:<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> Thanks Jon,<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> Here is what happens when I try this from<br>> >>>>>>>> communicado:<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> [davidk@communicado ~]$ myproxy-logon -l<br>> >>>>>>>>>>>>>> dkelly<br>> >>>>>>>> -s<br>> >>>>>>>>>>>>>> myproxy.teragrid.org<br>> >>>>>>>>>>>>>> Enter MyProxy pass phrase:<br>> >>>>>>>>>>>>>> A credential has been received for user dkelly<br>> >>>>>>>> in<br>> >>>>>>>>>>>>>> /tmp/x509up_u1878.<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> [davidk@communicado ranger]$ swift -sites.file<br>> >>>>>>>> sites.xml<br>> >>>>>>>>>>>>>> -tc.file<br>> >>>>>>>>>>>>>> tc.data 001-catsn-ranger.swift<br>> >>>>>>>>>>>>>> Swift svn swift-r4987 (swift modified locally)<br>> >>>>>>>> cog-r3229<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> RunID: 20110825-1326-o3e38fe0<br>> >>>>>>>>>>>>>> Progress: time: Thu, 25 Aug 2011 13:26:43<br>> >>>>>>>>>>>>>> -0500<br>> >>>>>>>>>>>>>> Progress: time: Thu, 25 Aug 2011 13:26:44<br>> >>>>>>>>>>>>>> -0500<br>> >>>>>>>> Selecting<br>> >>>>>>>>>>>>>> site:8<br>> >>>>>>>>>>>>>> Initializing site shared directory:2<br>> >>>>>>>>>>>>>> Execution failed:<br>> >>>>>>>>>>>>>> Authentication failed [Caused by: Failure<br>> >>>>>>>> unspecified at<br>> >>>>>>>>>>>>>> GSS-API<br>> >>>>>>>>>>>>>> level<br>> >>>>>>>>>>>>>> [Caused by: Unknown CA]]<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> Any ideas?<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> Thanks,<br>> >>>>>>>>>>>>>> David<br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> _______________________________________________<br>> >>>>>>>>>>>>>> Swift-devel mailing list<br>> >>>>>>>>>>>>>> Swift-devel@ci.uchicago.edu<br>> >>>>>>>>>>>>>><br>> >>>>>>>><br>> >>>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>><br>> >>>>>>>>>>>>>> --<br>> >>>>>>>>>>>>>> Ketan<br>> >>>>>>>>>>>>> _______________________________________________<br>> >>>>>>>>>>>>> Swift-devel mailing list<br>> >>>>>>>>>>>>> Swift-devel@ci.uchicago.edu<br>> >>>>>>>>>>>>><br>> >>>>>>>><br>> >>>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>> >>>>>>>><br>> >>>>>>>><br>> >>>>>>>> _______________________________________________<br>> >>>>>>>> Swift-devel mailing list<br>> >>>>>>>> Swift-devel@ci.uchicago.edu<br>> >>>>>>>><br>> >>>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>> >>>>>>>><br>> >>>>>>>><br>> >>>>>>>><br>> >>>>>>>><br>> >>>>>>>> --<br>> >>>>>>>> Sarah Kenny<br>> >>>>>>>> Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio<br>> >>>>>>>> Sci<br>> >>>>> III<br>> >>>>>>>> University of California Irvine, Dept. of Neurology<br>> >>>>>>>> ~<br>> >>>>> 773-818-8300<br>> >>>>>>>><br>> >>>>>>>> _______________________________________________<br>> >>>>>>>> Swift-devel mailing list<br>> >>>>>>>> Swift-devel@ci.uchicago.edu<br>> >>>>>>>><br>> >>>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br>> >>>>>>><br>> >>>>>><br>> >>>>>><br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>> --<br>> >>>>> Sarah Kenny<br>> >>>>> Programmer ~ Brain Circuits Laboratory ~ Rm 2224 Bio Sci III<br>> >>>>> University of California Irvine, Dept. of Neurology ~<br>> >>>>> 773-818-8300<br>> >>>>><br>> >>>><br>> >>>><br>> >>>> _______________________________________________<br>> >>>> Swift-devel mailing list<br>> >>>> Swift-devel@ci.uchicago.edu<br>> >>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br><br><br><br>_______________________________________________<br>Swift-devel mailing list<br>Swift-devel@ci.uchicago.edu<br>https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel<br></blockquote><br><span><br><br>-- <br><span name="x"></span>Michael Wilde<br>Computation Institute, University of Chicago<br>Mathematics and Computer Science Division<br>Argonne National Laboratory<br><span name="x"></span><br></span></div></div></div></body></html>