<div dir="ltr">Hi, <div><br></div><div>I've downloaded the package again and it worked just fine.</div><div><br></div><div>Thank you very much, Yadu and Mihael!</div><div><br></div><div>Igor</div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-05-05 16:27 GMT-03:00 Mihael Hategan <span dir="ltr"><<a href="mailto:hategan@mcs.anl.gov" target="_blank">hategan@mcs.anl.gov</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Have you modified any jar files or copied them from another swift<br>
package?<br>
<br>
The coaster bootstrap stores checksums of the jar files that it needs<br>
(calculated at swift compile time) and checks all jar files that come<br>
over an unsecured network against them. Maybe there should be a tool to<br>
update these checksums when needed, not just at compile time.<br>
<br>
Mihael<br>
<br>
On Tue, 2015-05-05 at 11:01 -0300, Igor Russo wrote:<br>
> Hi Mihael,<br>
<div class="HOEnZb"><div class="h5">><br>
> Sorry to bother you again.<br>
><br>
> You were right, after configuring the port forwarding the script is able to<br>
> connect.<br>
><br>
> But i still get an error "Checksum does not match".<br>
><br>
> Here goes the content of the ~/coaster-bootstrap-xxx.log file:<br>
><br>
> using plain mode<br>
> BS: <a href="http://189.12.232.9:50006" target="_blank">http://189.12.232.9:50006</a><br>
> which: no gmd5sum in<br>
> (/opt/openmpi/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/bio/ncbi/bin:/opt/bio/mpiblast/bin:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/tcoffee/bin:/opt/bio/hmmer/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gromacs/bin:/opt/bio/gmap/bin:/opt/bio/tigr/bin:/opt/bio/autodocksuite/bin:/opt/bio/wgs/bin:/opt/eclipse:/opt/ganglia/bin:/opt/ganglia/sbin:/usr/java/latest/bin:/opt/maven/bin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/condor/bin:/opt/condor/sbin:/opt/gridengine/bin/linux-x64)<br>
> Expected checksum: 9b7bd5a96a2912cf8d06d1a2fd891620<br>
> Computed checksum: 9b7bd5a96a2912cf8d06d1a2fd891620<br>
> JAVA=/usr/java/latest/bin/java<br>
> plain /usr/java/latest/bin/java -Djava=/usr/java/latest/bin/java -Xmx64M<br>
> -DGLOBUS_TCP_PORT_RANGE=<br>
> -DX509_USER_PROXY=/home/igor/.globus/sshproxy-1344874142-1432003400<br>
> -DX509_CERT_DIR=/home/igor/.globus/sshCAcert-1344874142-1432003400.pem<br>
> -DGLOBUS_HOSTNAME=<a href="http://cluster.mmc.ufjf.br" target="_blank">cluster.mmc.ufjf.br</a> -Duser.home=/home/igor -jar<br>
> /tmp/bootstrap.xTzo3v <a href="http://189.12.232.9:50006" target="_blank">http://189.12.232.9:50006</a> <a href="https://189.12.232.9:50005" target="_blank">https://189.12.232.9:50005</a><br>
> 11100954039<br>
> Failed to download cog-provider-coaster-0.3.jar:<br>
> java.lang.RuntimeException: Checksum does not match.<br>
><br>
><br>
> Thanks,<br>
> Igor<br>
><br>
> 2015-05-04 18:52 GMT-03:00 Mihael Hategan <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a>>:<br>
><br>
> ><br>
> > Hi,<br>
> ><br>
> > In most cases (globus, coasters), the service side (legion in this case)<br>
> > needs the ability to connect back to the client (your home connection).<br>
> ><br>
> > Correct me if I'm wrong, but you are on a DSL line, behind a router with<br>
> > NAT. If so, you must configure the router to forward some incoming<br>
> > connections to the actual machine from which you are running swift from.<br>
> > Typically this is done by configuring a certain port range forwarding on<br>
> > the router (Yadu suggested GLOBUS_TCP_PORT_RANGE=50000,51000, so that<br>
> > port range should be matched on the router).<br>
> ><br>
> > The gist of it is that swift starts a simple shell script on legion that<br>
> > downloads a small java app from the client side and launches it. Said<br>
> > shell script logs things into ~/coaster-bootstrap-xxx.log files. The<br>
> > contents of the bootstrap logs is probably very useful here.<br>
> ><br>
> > If all of that goes well, the aforementioned small java app downloads<br>
> > the full coaster service from the client and starts it. Once started,<br>
> > the coaster service connects back to Swift. The last two parts log their<br>
> > doings in ~/.globus/coasters/*.log. Those can be useful, too, if they<br>
> > exist.<br>
> ><br>
> > Mihael<br>
> ><br>
> > On Mon, 2015-05-04 at 18:27 -0300, Igor Russo wrote:<br>
> > > Hi Yadu,<br>
> > ><br>
> > > Yes, i can ssh from my laptop to the cluster directly.<br>
> > ><br>
> > > The coaster-bootstrap-*.log files are created in the remote system.<br>
> > ><br>
> > > I'm sending the log file attached.<br>
> > ><br>
> > > Thanks,<br>
> > > Igor<br>
> > ><br>
> > > 2015-05-04 16:57 GMT-03:00 Yadu Nand Babuji <<a href="mailto:yadunand@uchicago.edu">yadunand@uchicago.edu</a>>:<br>
> > ><br>
> > > > Hi Igor,<br>
> > > ><br>
> > > > Are you able to ssh from your machine to legion directly without<br>
> > entering<br>
> > > > passwords ?<br>
> > > > Could you please send us a tarball of the runNNN directories for a<br>
> > failing<br>
> > > > run ?<br>
> > > ><br>
> > > > I've put the following settings in my ~/.ssh/config on my laptop and<br>
> > setup<br>
> > > > ssh keys on<br>
> > > > both socrates and legion. This allows me to use "ssh<br>
> > <a href="http://legion.rc.ucl.ac.uk" target="_blank">legion.rc.ucl.ac.uk</a>"<br>
> > > > and connect.<br>
> > > ><br>
> > > > Host <a href="http://legion.rc.ucl.ac.uk" target="_blank">legion.rc.ucl.ac.uk</a><br>
> > > > User YOUR_USERNAME<br>
> > > > Hostname <a href="http://legion.rc.ucl.ac.uk" target="_blank">legion.rc.ucl.ac.uk</a><br>
> > > > ProxyCommand ssh socrates -W %h:%p<br>
> > > ><br>
> > > > Host socrates<br>
> > > > Hostname <a href="http://socrates.ucl.ac.uk" target="_blank">socrates.ucl.ac.uk</a><br>
> > > > User YOUR_USERNAME<br>
> > > > ForwardAgent yes<br>
> > > ><br>
> > > > Thanks,<br>
> > > > Yadu<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > On 05/04/2015 07:51 AM, Igor Russo wrote:<br>
> > > ><br>
> > > > Hi Yadu,<br>
> > > ><br>
> > > > Thanks again.<br>
> > > ><br>
> > > > I tried your suggestion. Now i'm not getting the previous error, but<br>
> > the<br>
> > > > jobs aren't being submitted:<br>
> > > ><br>
> > > > RunID: run001<br>
> > > > Progress: Seg, 04 Mai 2015 09:32:54-0300<br>
> > > > Progress: Seg, 04 Mai 2015 09:32:55-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:33:25-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:33:55-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:34:25-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:34:55-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:35:25-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:35:55-0300 Submitting:1<br>
> > > > Progress: Seg, 04 Mai 2015 09:36:25-0300 Submitting:1<br>
> > > ><br>
> > > > In the the log file, i notice the following errors:<br>
> > > ><br>
> > > > 2015-05-04 09:24:06,223-0300 INFO ServiceManager Service does not<br>
> > > > appear to be registered with this manager<br>
> > > > 2015-05-04 09:24:06,223-0300 INFO ServiceManager Coaster service<br>
> > ended.<br>
> > > > Reason: null<br>
> > > ><br>
> > > > Thanks,<br>
> > > > Igor<br>
> > > ><br>
> > > ><br>
> > > > 2015-05-01 17:47 GMT-03:00 Yadu Nand Babuji <<a href="mailto:yadunand@uchicago.edu">yadunand@uchicago.edu</a>>:<br>
> > > ><br>
> > > >> Hi Igor,<br>
> > > >><br>
> > > >> The remote connection system requires that the local machine you run<br>
> > > >> the swift client on has<br>
> > > >> a public ip address. It looks like swift was not able to guess it and<br>
> > set<br>
> > > >> it to <a href="http://igor-ubuntu:51251" target="_blank">http://igor-ubuntu:51251</a><br>
> > > >><br>
> > > >> Could you retry running part04 after doing the next step, and please<br>
> > > >> make sure your environment has<br>
> > > >> these variables set whenever you run swift to remote systems :<br>
> > > >> export GLOBUS_HOSTNAME=<PUBLIC_IP_OF_YOUR_MACHINE><br>
> > > >> export GLOBUS_TCP_PORT_RANGE=50000,51000<br>
> > > >><br>
> > > >> Thanks,<br>
> > > >> Yadu<br>
> > > >><br>
> > > >><br>
> > > >> On 05/01/2015 02:29 PM, Igor Russo wrote:<br>
> > > >><br>
> > > >> Hi Yadu,<br>
> > > >><br>
> > > >> Thank you very much!<br>
> > > >><br>
> > > >> I changed the config file with the data from my cluster.<br>
> > > >><br>
> > > >> When executing the 4th part of Swift-tutorial, i'm getting the<br>
> > > >> following error:<br>
> > > >> "Failed to download bootstrap jar from ..."<br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > >><br>
> > --------------------------------------------------------------------------------<br>
> > > >><br>
> > > >> RunID: run031<br>
> > > >> Progress: Sex, 01 Mai 2015 15:40:42-0300<br>
> > > >> Progress: Sex, 01 Mai 2015 15:40:43-0300 Submitting:1<br>
> > > >><br>
> > > >> Execution failed:<br>
> > > >> Exception in sort:<br>
> > > >> Arguments: [-n, unsorted.txt]<br>
> > > >> Host: mmc<br>
> > > >> Directory: p4-run031/jobs/s/sort-go28d68m<br>
> > > >> exception @ swift-int-staging.k, line: 165<br>
> > > >> Caused by:<br>
> > > >> exception @ swift-int-staging.k, line: 160<br>
> > > >> Caused by: null<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Could<br>
> > > >> not submit job<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Could<br>
> > > >> not start coaster service<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Task<br>
> > > >> ended before registration was received.<br>
> > > >> Failed to download bootstrap jar from <a href="http://igor-ubuntu:51251" target="_blank">http://igor-ubuntu:51251</a><br>
> > > >><br>
> > > >> k:assign @ swift.k, line: 174<br>
> > > >> Caused by: Exception in sort:<br>
> > > >> Arguments: [-n, unsorted.txt]<br>
> > > >> Host: mmc<br>
> > > >> Directory: p4-run031/jobs/s/sort-go28d68m<br>
> > > >> exception @ swift-int-staging.k, line: 165<br>
> > > >> Caused by:<br>
> > > >> exception @ swift-int-staging.k, line: 160<br>
> > > >> Caused by: null<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Could<br>
> > > >> not submit job<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Could<br>
> > > >> not start coaster service<br>
> > > >> Caused by:<br>
> > > >> org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:<br>
> > Task<br>
> > > >> ended before registration was received.<br>
> > > >> Failed to download bootstrap jar from <a href="http://igor-ubuntu:51251" target="_blank">http://igor-ubuntu:51251</a><br>
> > > >><br>
> > > >><br>
> > > >><br>
> > --------------------------------------------------------------------------------<br>
> > > >><br>
> > > >> Thanks,<br>
> > > >> Igor<br>
> > > >><br>
> > > >> 2015-05-01 13:47 GMT-03:00 Yadu Nand Babuji <<a href="mailto:yadunand@uchicago.edu">yadunand@uchicago.edu</a>>:<br>
> > > >><br>
> > > >>> Hi Igor,<br>
> > > >>><br>
> > > >>> Swift does support SGE clusters, and you can refer to the<br>
> > swift-tutorial<br>
> > > >>> for sample code and configurations from this link:<br>
> > > >>> <a href="https://github.com/swift-lang/swift-tutorial" target="_blank">https://github.com/swift-lang/swift-tutorial</a><br>
> > > >>><br>
> > > >>> Here's a sample config from our test-suite for Godzilla, an SGE<br>
> > cluster<br>
> > > >>> at UChicago:<br>
> > > >>><br>
> > > >>><br>
> > <a href="https://github.com/swift-lang/swift-k/blob/master/tests/sites/godzilla/swift.conf" target="_blank">https://github.com/swift-lang/swift-k/blob/master/tests/sites/godzilla/swift.conf</a><br>
> > > >>> You could modify and add this config to the swift.conf file in the<br>
> > > >>> swift-tutorial to run<br>
> > > >>> Swift on any machine and execute on a remote SGE cluster.<br>
> > > >>><br>
> > > >>> SGE is a widely used resource manager and most sites have<br>
> > differences in<br>
> > > >>> their setups that make each site unique. If you run into issues with<br>
> > the<br>
> > > >>> default<br>
> > > >>> swift package, and could provide help in figuring out specifics of<br>
> > your<br>
> > > >>> cluster, we<br>
> > > >>> will help you adapt the Swift SGE provider to support your cluster.<br>
> > > >>><br>
> > > >>> Thanks,<br>
> > > >>> Yadu<br>
> > > >>><br>
> > > >>><br>
> > > >>><br>
> > > >>> On 04/28/2015 05:09 PM, Igor Russo wrote:<br>
> > > >>><br>
> > > >>> Hi All,<br>
> > > >>><br>
> > > >>> It is possible to use Swift with a remote SGE/OGE cluster?<br>
> > > >>><br>
> > > >>> Regards,<br>
> > > >>> Igor<br>
> > > >>><br>
> > > >>><br>
> > > >>> _______________________________________________<br>
> > > >>> Swift-user mailing listSwift-user@ci.uchicago.eduhttps://<br>
> > <a href="http://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > >>><br>
> > > >>><br>
> > > >>><br>
> > > >>> _______________________________________________<br>
> > > >>> Swift-user mailing list<br>
> > > >>> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
> > > >>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > >>><br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > >> _______________________________________________<br>
> > > >> Swift-user mailing listSwift-user@ci.uchicago.eduhttps://<br>
> > <a href="http://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > >> _______________________________________________<br>
> > > >> Swift-user mailing list<br>
> > > >> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
> > > >> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > >><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > _______________________________________________<br>
> > > > Swift-user mailing listSwift-user@ci.uchicago.eduhttps://<br>
> > <a href="http://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > ><br>
> > > ><br>
> > > ><br>
</div></div><span class="im HOEnZb">> > > > _______________________________________________<br>
> > > > Swift-user mailing list<br>
> > > > <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
> > > > <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> > > ><br>
> > > _______________________________________________<br>
> > > Swift-user mailing list<br>
> > > <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
> > > <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> ><br>
> ><br>
> ><br>
> ><br>
</span><div class="HOEnZb"><div class="h5">> _______________________________________________<br>
> Swift-user mailing list<br>
> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
<br>
<br>
</div></div></blockquote></div><br></div>