[Swift-devel] Problem with 0.92 sending jobs to OSG via Condor-G

Thu Jan 13 08:45:48 CST 2011

This is very strange. I diffed the entire cog and modules/swift dir trees between my 0.92 and trunk working copies. The only changes I see are:
- my (one) local mod to remove Time:HiRes for Ranger in worker.pl
- David's patch to override (which *does* touch JobSpecification, suspiciously!

The latter I will investigate, but Im very surprised to see so few differences between 0.92 and a trunk working copy.

Mihael, I *thought* you integrated changes from stable-branch ( ~ 0.91) into the 0.92 branch. Or did you integrate those into trunk, and perhaps I took a later copy of trunk?  If you did indeed integrate the stable changes into trunk, *and* its possible that what I test on was not re-built after I did an svn update some time in the past few weeks, *then* perhaps the Condor bug crept in between recent changes to trunk.

Looking at svn log on my trunk copy, it seems almost as if one of these two adjacent revisions are somehow reverted or suddenly not working:
------------------------------------------------------------------------
r2021 | b_z_c | 2008-05-16 10:01:46 -0400 (Fri, 16 May 2008) | 1 line

JDK1.4.2 compatible string mangling
------------------------------------------------------------------------
r2020 | b_z_c | 2008-05-16 08:04:01 -0400 (Fri, 16 May 2008) | 1 line

double-quote symbols in arguments are now escaped
------------------------------------------------------------------------

I will also try to back off the Override patch and compile with 1.6.  I cant understand how my trunk got compiled, previously, without the override patch.

So I'll try to check my update vs build dates, how I did my older (working) build, 

Let me know if you have other ideas of how to diagnose this.

- Mike

----- Original Message -----
> So far I see no diffs between 0.92 swift/libexec and the trunk working
> copy I was using on engage-submit.
> 
> Where else should I look?
> 
> (Its very possible Im missing something; but I tried creating an
> artificial change in at least one case, a few dirs down, and svn diff
> picks it up)
> 
> - Mike
> 
> ----- Original Message -----
> > ----- Original Message -----
> > > > I need to check what local mods I had applied, but I think its
> > > > more
> > > > likely that some Condor submit file quoting fix fell off in 0.92
> > > > integration.
> > >
> > > Yeah. A svn diff > somefile would help.
> >
> > Hmmm. So far svn diffs show no changes within provider-condor,
> > neither
> > between trunk and 0.92 branch nor within my working copies of those
> > two on engage-submit, which seem to behave differently regarding
> > Condor quoting.
> >
> > Could the change(s) that were made a long time ago to fix Condor
> > quoting be in a different module than provider-condor? If so, whats
> > a
> > likely place to look?
> >
> > I'll check vdl-int.k next.
> >
> > - Mike
> >
> > > >
> > > > So Marc, sorry - this release is not usable for you yet.
> > > >
> > > > - Mike
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > Im trying my first tests of 0.92 on engage-submit, sending 100
> > > > > trivial
> > > > > cat jobs to 10 OSG sites.
> > > > >
> > > > > My jobs seem to be all dying with the error "Found illegal
> > > > > unescaped
> > > > > double-quote" (see below).
> > > > >
> > > > > Has anyone successfully run a Condor-G job on OSG with 0.92?
> > > > >
> > > > > I'll dig deeper and try the same test with the older version
> > > > > of
> > > > > trunk
> > > > > that Marc has been using here with better success. Will also
> > > > > try
> > > > > a
> > > > > single job run and capture a simpler log and the condor-g
> > > > > submit
> > > > > file.
> > > > >
> > > > > Allan, have you tried 0.92 against COndor-G? If not, could
> > > > > you?
> > > > >
> > > > > Sarah, we should add some Condor-G-to-GT2 testing to 0.92
> > > > > validation I
> > > > > think.
> > > > >
> > > > > - Mike
> > > > >
> > > > > --
> > > > >
> > > > > Caused by:
> > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException:
> > > > > Cannot submit job: Could not submit job (condor_submit
> > > > > reported
> > > > > an
> > > > > exit code of 1). Submitting job(s)
> > > > > Found illegal unescaped double-quote: "" -e /bin/cat -out
> > > > > outdir/f.0065.out -err stderr.txt -i -d outdir -if data.txt
> > > > > -of
> > > > > outdir/f.0065.out -k "" -cdmfile "" -status file -a
> > > > > data.txtThe
> > > > > full
> > > > > arguments you specified were:
> > > > > /osg/data/engage/tmp/osg.hpc.ufl.edu/catsn-20110113-0025-vv4p4up3/shared/_swiftwrap
> > > > > cat-ajxnee4k -jobdir a -scratch "" -e /bin/cat -out
> > > > > outdir/f.0065.out
> > > > > -err stderr.txt -i -d outdir -if data.txt -of
> > > > > outdir/f.0065.out
> > > > > -k
> > > > > ""
> > > > > -cdmfile "" -status file -a data.txt
> > > > >
> > > > >
> > > > > Script is:
> > > > >
> > > > > e$ cat catsn.swift
> > > > > type file;
> > > > >
> > > > > app (file o) cat (file i)
> > > > > {
> > > > > cat @i stdout=@o;
> > > > > }
> > > > >
> > > > > file out[]<simple_mapper; location="outdir",
> > > > > prefix="f.",suffix=".out">;
> > > > > foreach j in [1:@toint(@arg("n","1"))] {
> > > > > file data<"data.txt">;
> > > > > out[j] = cat(data);
> > > > > }
> > > > >
> > > > >
> > > > > --
> > > > > Michael Wilde
> > > > > Computation Institute, University of Chicago
> > > > > Mathematics and Computer Science Division
> > > > > Argonne National Laboratory
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > >
> >
> > --
> > Michael Wilde
> > Computation Institute, University of Chicago
> > Mathematics and Computer Science Division
> > Argonne National Laboratory
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> --
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory