[Swift-devel] gridFTP urls for data in provider staging mode
Michael Wilde
wilde at mcs.anl.gov
Thu Nov 17 10:49:17 CST 2011
(moving this thread to the swift-devel list)
Mihael, can you clarify a few more aspects of this?
- the staging mode determines if the file is read by the client ("file") or service ("proxy"), correct? Both should work?
- when a gsiftp-protocol URI is read, is it read and sent to the worker in a read loop, with good-sized buffers? Or is it read into local disk and then sent to the worker? Im hoping the former, so we can handle large files more reasonably.
- the provider used to read the file is determined by the protocol field in the URI, and not by the data provider listed for the site the job is going to, right? The latter is used solely for accessing the workdirectory, and hence does not apply (ie is not needed or ignored) for the case of provider staging, right?
Mike
----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "Mihael Hategan" <hategan at mcs.anl.gov>
> Cc: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> Sent: Thursday, November 17, 2011 8:38:15 AM
> Subject: Re: gridFTP urls for data in provider staging mode
> Mihael, I dont understand. Did you mean to say that the *service* wont
> accept a URL for which it cant open a stream?
>
> So how did this work for you? Did you not do a similar test to what
> Ketan is doing?
>
> (Ie, map a scalar file-type variable to a gsiftp:// URI?)
>
> And I thought the code tried to read the input file via a provider
> thats based on the protocol in the mapped file name's URI??? As
> opposed to assuming that its a local file and opening a Java stream on
> that name.
>
> Can you clarify what you tested and why you think Ketan's test is
> failing?
>
> Thats why I asked you to send us the test you ran: so we could
> replicate it to verify that we had all the needed code pieces.
>
> Thanks,
>
> - Mike
>
>
> ----- Original Message -----
> > From: "Mihael Hategan" <hategan at mcs.anl.gov>
> > To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> > Cc: "Michael Wilde" <wilde at mcs.anl.gov>
> > Sent: Wednesday, November 16, 2011 8:33:39 PM
> > Subject: Re: gridFTP urls for data in provider staging mode
> > That's a completely different issue.
> >
> > It seems that URL won't accept a URL for which it cannot open a
> > stream.
> > I will change the parsing.
> >
> > On Wed, 2011-11-16 at 17:03 -0600, Ketan Maheshwari wrote:
> > > Mihael,
> > >
> > > I think in this latest run I have got the correct
> > > CoGResourceIOProvider. The log is:
> > >
> > > http://ci.uchicago.edu/~ketan/worker-tmp.log
> > >
> > > And, this line matches:
> > > at
> > > org.globus.cog.abstraction.impl.file.coaster.handlers.providers.CoGResourceIOProvider.buildService(CoGResourceIOProvider.java:55)
> > > the file you sent.
> > >
> > >
> > > The error message is also pertaining to the gsiftp protocol:
> > > 2011/11/16 16:57:18.743 DEBUG 000000 1321484238602 getFileCBDataIn
> > > FAILED 520 Error staging in file:
> > > org.globus.cog.karajan.workflow.service.ProtocolException:
> > > org.globus.cog.karajan.workflow.service.ProtocolException:
> > > java.net.MalformedURLException: unknown protocol: gsiftp
> > >
> > > Regards,
> > > Ketan
> > >
> > >
> > > On Wed, Nov 16, 2011 at 3:12 PM, Mihael Hategan
> > > <hategan at mcs.anl.gov>
> > > wrote:
> > > On Wed, 2011-11-16 at 14:58 -0600, Michael Wilde wrote:
> > >
> > > > My theory as that for some reason the Java code in the
> > > service is not
> > > > recognizing the mapped URL as a gridftp URL, and is
> > > > hence
> > > treating the
> > > > gridftp URI like a file.
> > >
> > >
> > > The stack trace that Ketan sees occurs in a line that does
> > > not
> > > have
> > > anything relevant in the version of CoGResourceIOProvider
> > > that
> > > I sent.
> > >
> > > If you look at the worker log, you see this:
> > >
> > > org.globus.cog.abstraction.impl.file.coaster.handlers.providers.CoGResourceIOProvider$Reader.<init>(CoGResourceIOProvider.java:143)
> > >
> > > If you look at the CoGResourceIOProvider I sent:
> > >
> > > 142 public void resume() {
> > > 143 } <----
> > >
> > > The original, however, has the following:
> > > 142 this.cb = cb;
> > > 143 fc = new FileInputStream(f).getChannel(); <----
> > >
> > > So it's pretty clear to me that whatever Ketan is running
> > > is
> > > using the
> > > old file.
> > >
> > > That's a build problem of some sort. I suggest checking in
> > > the
> > > two files
> > > into the 0.93.1 branch and then doing a clean build from
> > > that
> > > branch as
> > > well as making sure that no helper scripts that modify the
> > > standard
> > > environment are used.
> > >
> > >
> > > >
> > > > The log should help us see if we running the same swift
> > > > as
> > > you, and
> > > > where the patterns diverge.
> > > >
> > > > I really hope that we can resolve this expeditiously.
> > > >
> > > > - Mike
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Mihael Hategan" <hategan at mcs.anl.gov>
> > > > > To: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
> > > > > Cc: "Michael Wilde" <wilde at mcs.anl.gov>
> > > > > Sent: Wednesday, November 16, 2011 11:50:09 AM
> > > > > Subject: Re: gridFTP urls for data in provider staging
> > > mode
> > > > > There was nothing special in them.
> > > > >
> > > > > I used proxy mode (though it shouldn't matter) and, of
> > > course,
> > > > > provider
> > > > > staging.
> > > > >
> > > > > I generally helps to make it simple and use automatic
> > > coasters and
> > > > > start
> > > > > with local:local.
> > > > >
> > > > > On Wed, 2011-11-16 at 10:55 -0600, Ketan Maheshwari
> > > > > wrote:
> > > > > > Mihael,
> > > > > >
> > > > > >
> > > > > > Could you send your sites.xml, cf and source files
> > > > > > with
> > > which you
> > > > > > tested this.
> > > > > >
> > > > > > Regards,
> > > > > > Ketan
> > > > > >
> > > > > > On Wed, Nov 16, 2011 at 10:19 AM, Ketan Maheshwari
> > > > > > <ketancmaheshwari at gmail.com> wrote:
> > > > > > I checked which coaster-service I am using
> > > > > > and
> > > think it is
> > > > > > coming from the right location in 0.93:
> > > > > >
> > > > > >
> > > > > > [communicado:swiftgrid]$ which
> > > > > > coaster-service
> > > > > >
> > > ~/swift-install/0.93/cog/modules/swift/dist/swift-svn/bin/coaster-service
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Nov 15, 2011 at 4:15 PM, Mihael
> > > > > > Hategan
> > > > > > <hategan at mcs.anl.gov> wrote:
> > > > > > Also, this is not an issue with
> > > worker.pl. It's
> > > > > > really
> > > > > > the java part.
> > > > > > But it's the one used by the coaster
> > > service, so
> > > > > > make
> > > > > > sure that the
> > > > > > coaster service you are starting is
> > > > > > from
> > > a tree with
> > > > > > the updated
> > > > > > CoGResourceIOProvider.java.
> > > > > >
> > > > > > On Tue, 2011-11-15 at 15:20 -0600,
> > > > > > Ketan
> > > Maheshwari
> > > > > > wrote:
> > > > > >
> > > > > >
> > > > > > > I am using passive coasters:
> > > > > > >
> > > > > > >
> > > > > > > coaster-service -port 1984
> > > > > > > -localport
> > > 35753 -nosec
> > > > > > -passive
> > > > > > >
> > > > > > >
> > > > > > > but I will run once again putting
> > > > > > > the
> > > modified
> > > > > > worker.pl in source
> > > > > > > tree.
> > > > > > >
> > > > > > > On Tue, Nov 15, 2011 at 3:18 PM,
> > > Michael Wilde
> > > > > > <wilde at mcs.anl.gov>
> > > > > > > wrote:
> > > > > > > If this is automatic
> > > > > > > coasters,
> > > you need to
> > > > > > change worler.pl in
> > > > > > > the source tree and re ant
> > > dist: its
> > > > > > included in the automatic
> > > > > > > boot process and not
> > > > > > > fetched
> > > from bin/
> > > > > > >
> > > > > > > - Mike
> > > > > > >
> > > > > > > ----- Original Message
> > > > > > > -----
> > > > > > > > From: "Ketan Maheshwari"
> > > > > > <ketancmaheshwari at gmail.com>
> > > > > > > > To: "Mihael Hategan"
> > > > > > > > <hategan at mcs.anl.gov>
> > > > > > > > Cc: "Michael Wilde"
> > > <wilde at mcs.anl.gov>
> > > > > > >
> > > > > > > > Sent: Tuesday, November
> > > > > > > > 15,
> > > 2011
> > > > > > > > 12:53:19
> > > > > > PM
> > > > > > > > Subject: Re: gridFTP
> > > > > > > > urls
> > > for data in
> > > > > > provider staging mode
> > > > > > >
> > > > > > >
> > > > > > > > Mihael,
> > > > > > > >
> > > > > > > >
> > > > > > > > I tried to match the
> > > > > > > > file
> > > you sent that
> > > > > > > > I
> > > > > > copied into the
> > > > > > > swift tree
> > > > > > > > using sum and diff:
> > > > > > > >
> > > > > > > >
> > > > > > > > [login:cog]$ sum
> > > > > > ~/CoGResourceIOProvider.java
> > > > > > > > 63915 8
> > > > > > > > [login:cog]$ sum
> > > > > > >
> > > > > >
> > > > ./modules/provider-coaster/src/org/globus/cog/abstraction/impl/file/coaster/handlers/providers/CoGResourceIOProvider.java
> > > > > > > > 63915 8
> > > > > > > > [login:cog]$
> > > > > >
> > > diff /home/ketan/CoGResourceIOProvider.java
> > > > > > >
> > > > > >
> > > > ./modules/provider-coaster/src/org/globus/cog/abstraction/impl/file/coaster/handlers/providers/CoGResourceIOProvider.java
> > > > > > > >
> > > > > > > >
> > > > > > > > I recompiled swift. And
> > > replaced the
> > > > > > worker.pl in
> > > > > > > dist/swift-svn/bin
> > > > > > > > with the one you sent.
> > > > > > > >
> > > > > > > >
> > > > > > > > I made sure that I am
> > > running the
> > > > > > > > correct
> > > > > > swift:
> > > > > > > >
> > > > > > > > [communicado:swiftgrid]$
> > > which swift
> > > > > > > >
> > > > > > >
> > > > > >
> > > ~/swift-install/0.93/cog/modules/swift/dist/swift-svn/bin/swift
> > > > > > > >
> > > > > > > >
> > > > > > > > I tried the example
> > > > > > > > again
> > > and it is
> > > > > > > > giving
> > > > > > the same failure
> > > > > > > message.
> > > > > > > >
> > > > > > > > On Tue, Nov 15, 2011 at
> > > > > > > > 2:19
> > > PM, Mihael
> > > > > > Hategan <
> > > > > > > hategan at mcs.anl.gov
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > The error in the worker
> > > > > > > > logs
> > > can only
> > > > > > appear in the original
> > > > > > > > CoGResourceIOProvider,
> > > > > > > > not
> > > the one I
> > > > > > > > sent
> > > > > > you. Make sure you
> > > > > > > got that
> > > > > > > > in
> > > > > > > > the right place, that
> > > > > > > > you
> > > recompiled,
> > > > > > > > and
> > > > > > that you are using
> > > > > > > the
> > > > > > > > modified swift.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, 2011-11-15 at
> > > > > > > > 11:23
> > > -0600, Ketan
> > > > > > Maheshwari wrote:
> > > > > > > > > Mike,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The worker log is
> > > > > > > > > here:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > >
> > > http://ci.uchicago.edu/~ketan/worker-catsn-gridftp.log
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The problem looks like
> > > > > > > > > the
> > > gridFTP
> > > > > > > > > path
> > > > > > is not being
> > > > > > > properly
> > > > > > > > > captured
> > > > > > > > > from this line:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 2011/11/08
> > > > > > > > > 16:14:42.650
> > > DEBUG 000000
> > > > > > 1320790482491
> > > > > > > > > getFileCBDataInIndirect
> > > error:
> > > > > > > > >
> > > > > >
> > > org.globus.cog.karajan.workflow.service.ProtocolException:
> > > > > > > > >
> > > java.io.FileNotFoundException:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > /scratch/local/ketan/catsn-gridftp-staging-method-file/./gsiftp:/
> > > > > > > > >
> > > > > > >
> > > > > >
> > > gridftp.pads.ci.uchicago.edu/gpfs/pads/swift/ketan/data0000.txt
> > > > > > (No
> > > > > > > > > such file or
> > > > > > > > > directory)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Ketan
> > > > > > > > >
> > > > > > > > > On Tue, Nov 15, 2011
> > > > > > > > > at
> > > 10:42 AM,
> > > > > > Michael Wilde <
> > > > > > > wilde at mcs.anl.gov
> > > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > Ketan, the 520 error
> > > > > > > > > means
> > > that the
> > > > > > worker.pl script died.
> > > > > > > > >
> > > > > > > > > Can you run again, and
> > > capture all the
> > > > > > logs: swift log,
> > > > > > > > > coaster log (if
> > > > > > > > > separate),
> > > and worker
> > > > > > log (with DEBUG set,
> > > > > > > > > possibly important).
> > > > > > > > > Maybe
> > > you can
> > > > > > > > > trakc
> > > > > > back from the 520
> > > > > > > > > error. The worker log
> > > should give some
> > > > > > insight into why
> > > > > > > the
> > > > > > > > > 520 is occurring. I
> > > > > > > > > cant
> > > recall what
> > > > > > > > > the
> > > > > > # means but its a
> > > > > > > > > common one. Probably
> > > "failed to stage
> > > > > > file", in which case
> > > > > > > the
> > > > > > > > > worker log may help
> > > clarify why.
> > > > > > > > >
> > > > > > > > > - Mike
> > > > > > > > >
> > > > > > > > > ----- Original Message
> > > -----
> > > > > > > > >
> > > > > > > > > > From: "Ketan
> > > > > > > > > > Maheshwari"
> > > <
> > > > > > ketancmaheshwari at gmail.com >
> > > > > > > > > > To: "Mihael Hategan"
> > > > > > > > > > <
> > > > > > hategan at mcs.anl.gov >
> > > > > > > > > > Cc: "Michael Wilde"
> > > > > > > > > > <
> > > > > > wilde at mcs.anl.gov >
> > > > > > > > > > Sent: Tuesday,
> > > > > > > > > > November
> > > 15, 2011
> > > > > > 8:15:48 AM
> > > > > > > > > > Subject: Re: gridFTP
> > > urls for data
> > > > > > > > > > in
> > > > > > provider staging
> > > > > > > mode
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Hi Mihael,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I rebuilt swift with
> > > these files
> > > > > > included but could not
> > > > > > > get
> > > > > > > > > the test
> > > > > > > > > > run successfully. I
> > > tried three
> > > > > > > > > > tests,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1. catsn without
> > > gridftp: worked
> > > > > > > > > > 2. catsn with
> > > > > > > > > > gridftp
> > > staging method
> > > > > > file: fails
> > > > > > > > > > 3. catsn with
> > > > > > > > > > gridftp
> > > staging method
> > > > > > proxy: fails
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Both failures seem
> > > > > > > > > > to
> > > > > > > > > > be
> > > related to
> > > > > > the staging from the
> > > > > > > > > following
> > > > > > > > > > stderr message:
> > > > > > > > > >
> > > > > > > > > > Caused by: null
> > > > > > > > > > Caused by:
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > org.globus.cog.abstraction.impl.common.execution.JobException:
> > > > > > > > > Job
> > > > > > > > > > failed with an exit
> > > > > > > > > > code
> > > of 520
> > > > > > > > > > Final status: time:
> > > > > > > > > > Tue,
> > > 15 Nov 2011
> > > > > > 10:08:37 -0600
> > > > > > > Failed:1
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > The logs are as
> > > > > > > > > > follows:
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > http://ci.uchicago.edu/~ketan/catsn-gridftp-staging-method-file.log
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > http://ci.uchicago.edu/~ketan/catsn-gridftp-staging-method-proxy.log
> > > > > > > > > >
> > > > > >
> > > http://ci.uchicago.edu/~ketan/catsn-nogridftp.log
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Ketan
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 14, 2011
> > > > > > > > > > at
> > > 1:47 PM,
> > > > > > Mihael Hategan <
> > > > > > > > > hategan at mcs.anl.gov
> > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, 2011-11-14
> > > > > > > > > > at
> > > 08:39 -0600,
> > > > > > Michael Wilde wrote:
> > > > > > > > > > > Mihael, I see that
> > > > > > > > > > > you
> > > made the
> > > > > > branch for this. Did
> > > > > > > you
> > > > > > > > > commit it?
> > > > > > > > > > > If not, can you?
> > > > > > > > > >
> > > > > > > > > > Not yet. I'm a bit
> > > > > > > > > > busy
> > > this week
> > > > > > seeing that I seem to
> > > > > > > have
> > > > > > > > > no idea
> > > > > > > > > > how
> > > > > > > > > > to do any of the
> > > homework (which
> > > > > > > > > > isn't
> > > > > > really a new
> > > > > > > thing),
> > > > > > > > > but I'll
> > > > > > > > > > try
> > > > > > > > > > to commit the input
> > > stuff as soon as
> > > > > > > > > > I
> > > > > > can.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Whats needed for
> > > testing the
> > > > > > > > > > > output
> > > > > > side as well?
> > > > > > > > > >
> > > > > > > > > > Should be equally
> > > > > > > > > > easy.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'd like to push
> > > forward with
> > > > > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > > Yes, I know, I'm
> > > > > > > > > > sorry.
> > > I'll get to
> > > > > > > > > > it
> > > > > > as soon as I can.
> > > > > > > > > >
> > > > > > > > > > Well, you know what,
> > > > > > > > > > I'm
> > > attaching
> > > > > > > > > > the
> > > > > > modified relevant
> > > > > > > > > files. Just
> > > > > > > > >
> > > > > > > > > > rename worker.pl.pr
> > > > > > > > > > to
> > > worker.pl .
> > > > > > Feel free to commit
> > > > > > > them
> > > > > > > > > to the new
> > > > > > > > > > branch.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Ketan
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Michael Wilde
> > > > > > > > > Computation Institute,
> > > University of
> > > > > > Chicago
> > > > > > > > > Mathematics and
> > > > > > > > > Computer
> > > Science
> > > > > > Division
> > > > > > > > > Argonne National
> > > Laboratory
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Ketan
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Ketan
> > > > > > >
> > > > > > > --
> > > > > > > Michael Wilde
> > > > > > > Computation Institute,
> > > University of
> > > > > > > Chicago
> > > > > > > Mathematics and Computer
> > > Science Division
> > > > > > > Argonne National
> > > > > > > Laboratory
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Ketan
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Ketan
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Ketan
> > > > > >
> > > > > >
> > > > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Ketan
> > >
> > >
>
> --
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
--
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory
More information about the Swift-devel
mailing list