[Swift-devel] Example of wrapper staging?

[Michael Wilde]

Ok, that works great now, both with the conf you posted and with this one:

wrapperStagingLocalServer: "file://"

site.local {
     staging: "wrapper"
     scratch: "/tmp/wilde/swiftest"
}

Thanks,

- Mike

On 11/8/14 4:41 PM, Mihael Hategan wrote:
> (solved on IM but replying here for reference).
>
> I made a mistake in the swift.conf I sent. wrapperStagingLocalServer is
> a global option not a site option. However, it should work with the
> vanilla swift.conf posted, which had the wrong option commented out.
>
> Mihael
>
>
> On Sat, 2014-11-08 at 16:37 -0600, Michael Wilde wrote:
>> I get this error:
>>
>> org.griphyn.vdl.util.SwiftConfigException: swift.conf:4 unexpected
>> property 'site.local.wrapperStagingLocalServer'
>>       at
>> org.griphyn.vdl.util.SwiftConfigSchema.validate(SwiftConfigSchema.java:214)
>>
>> Ive tried spelling this property both as in your sample .conf and as in
>> your text:
>>
>> wrapperStagingLocalServer and wrapperStageingLocalServer
>>
>> Neither is accepted.  Did a schema change not get committed?
>>
>> For general info: where is the json schema file that a user can check to
>> find all valid conf parameters?
>>
>> Thanks,
>>
>> - Mike
>>
>>
>>
>> On 11/8/14 4:13 PM, Mihael Hategan wrote:
>>> type file;
>>>
>>> app (file c) cat(file a, file b) {
>>> 	cat filename(a) filename(b) stdout=filename(c);
>>> }
>>>
>>> file c <"c.txt">;
>>> file a <"a.txt">;
>>> file b <"http://mirror.internode.on.net/pub/test/1meg.test">;
>>>
>>> c = cat(a, b);
>>> --------------------------------------------------------
>>>
>>> You would need a proper swift.conf. Here's an example of that:
>>> --------------------------------------------------------
>>> site.local {
>>> 	staging: "wrapper"
>>> 	scratch: "/some/scratch/dir"
>>>           # *see email comments for details on the following option
>>> 	# wrapperStagingLocalServer:"file://"
>>> }
>>> --------------------------------------------------------
>>>
>>>
>>> With a proper scratch dir there, this*should*  work for you without any
>>> additional tweaking.
>>>
>>> Check _swiftwrap.wrapperstaging's stagein() and stageout() functions.
>>> That's where you would add additional protocols.
>>>
>>> The wrapperStagingLocalServer sets the default staging location for
>>> things that do not have an explicit scheme. In the case above, "c.txt"
>>> and "a.txt" are staged in and out using"file://". If you wanted the
>>> default to be some gridftp server, you would say something like
>>> 'wrapperStageingLocalServer: "gsiftp://gridftp.server.org/"'. Of course,
>>> _swiftwrap.wrapperstaging would need to know how to handle gsiftp URLs,
>>> which it doesn't at the moment.
>>>
>>> Mihael
>

-- 
Michael Wilde
Mathematics and Computer Science          Computation Institute
Argonne National Laboratory               The University of Chicago