[Swift-devel] Example of wrapper staging?

[Mihael Hategan]

(solved on IM but replying here for reference).

I made a mistake in the swift.conf I sent. wrapperStagingLocalServer is
a global option not a site option. However, it should work with the
vanilla swift.conf posted, which had the wrong option commented out.

Mihael


On Sat, 2014-11-08 at 16:37 -0600, Michael Wilde wrote:
> I get this error:
> 
> org.griphyn.vdl.util.SwiftConfigException: swift.conf:4 unexpected 
> property 'site.local.wrapperStagingLocalServer'
>      at 
> org.griphyn.vdl.util.SwiftConfigSchema.validate(SwiftConfigSchema.java:214)
> 
> Ive tried spelling this property both as in your sample .conf and as in 
> your text:
> 
> wrapperStagingLocalServer and wrapperStageingLocalServer
> 
> Neither is accepted.  Did a schema change not get committed?
> 
> For general info: where is the json schema file that a user can check to 
> find all valid conf parameters?
> 
> Thanks,
> 
> - Mike
> 
> 
> 
> On 11/8/14 4:13 PM, Mihael Hategan wrote:
> > type file;
> >
> > app (file c) cat(file a, file b) {
> > 	cat filename(a) filename(b) stdout=filename(c);
> > }
> >
> > file c <"c.txt">;
> > file a <"a.txt">;
> > file b <"http://mirror.internode.on.net/pub/test/1meg.test">;
> >
> > c = cat(a, b);
> > --------------------------------------------------------
> >
> > You would need a proper swift.conf. Here's an example of that:
> > --------------------------------------------------------
> > site.local {
> > 	staging: "wrapper"
> > 	scratch: "/some/scratch/dir"
> >          # *see email comments for details on the following option
> > 	# wrapperStagingLocalServer:"file://"
> > }
> > --------------------------------------------------------
> >
> >
> > With a proper scratch dir there, this*should*  work for you without any
> > additional tweaking.
> >
> > Check _swiftwrap.wrapperstaging's stagein() and stageout() functions.
> > That's where you would add additional protocols.
> >
> > The wrapperStagingLocalServer sets the default staging location for
> > things that do not have an explicit scheme. In the case above, "c.txt"
> > and "a.txt" are staged in and out using"file://". If you wanted the
> > default to be some gridftp server, you would say something like
> > 'wrapperStageingLocalServer: "gsiftp://gridftp.server.org/"'. Of course,
> > _swiftwrap.wrapperstaging would need to know how to handle gsiftp URLs,
> > which it doesn't at the moment.
> >
> > Mihael
>