[Swift-devel] plug points

[Mihael Hategan]

On Sun, 2007-04-29 at 17:00 +0100, Ben Clifford wrote:
> There are a number of places in swift where the 'core' production code 
> needs to interface with other code.
> 
> The most obvious of these is the interface for non-core mappers.
> 
> Other places seem to be:
> 
>   * job submission - whilst the rhetoric says 'we will use GRAM', its been 
> the case that we have used at least PBS and DeeF internally for some 
> projects that are not part of the core swift code; so it seems likely that 
> we should write down (and tidy up if necessary) the interface for plugging 
> in different job submission systems. I mostly know how PBS was connected 
> to Swift. I have no idea how DeeF was, and it would perhaps be 
> enlightening if someone would briefly describe how.

http://wiki.cogkit.org/index.php/Java_CoG_Kit_Abstraction_Guide
http://www.cogkit.org/release/current/api/abstraction-common/index.html

> 
>   * site selection - at least one group (the UBA people) have in part 
> neglected using swift because I haven't given them an answer about how to 
> do custom site selection.

Unfortunately this is not that well documented.
http://www.cogkit.org/viewcvs/viewcvs.cgi/src/cog/modules/karajan/src/org/globus/cog/karajan/scheduler/Scheduler.java?rev=HEAD&content-type=text/vnd.viewcvs-markup

> 
> It seems to me that we should formalise (to a lesser or greater extent) 
> how to plug thing in to the above interfaces - perhaps just a couple of 
> paragraphs on the recommended API, perhaps more. This is in part done for 
> mappers, in the tutorial.
> 
> At present, additions happen through a range of source level hacks - for 
> example, the mapper tutorial suggests as a final step 'now recompile Swift 
> to make your mapper work'; and there's a lookup table that PBS was 
> added to recently, despite not being a supported submission mechanism.

Regarding PBS, it works as pointed above. In principle, given a well
defined provider (a provider.properties file) on the classpath, it will
be available automatically.

> 
> The presence of a couple of pieces of hacked on cruft like this is not 
> really a problem; but its not good in the long term - it makes the 
> codebase as a whole less coherent, and confuses the Swift production code 
> with other (related but not the same) projects.

Right. Fortunately it's one piece that is quite formalized (at least the
abstraction stuff). The knowledge about it (conceptual link from Swift
to "that thing" is missing).

> 
> So one of the things I want to do, though not as a high priority, is 
> document the above and make them more coherent/sensible if necessary.
>