[Swift-devel] Swiftconfig merge / changes

[Sarah Kenny]

i did commit the latest version of meta.sh (sitetester) in python...so, we
know where my preference lies :)

On Tue, Feb 1, 2011 at 10:04 PM, jon.monette at gmail.com <
jon.monette at gmail.com> wrote:

> I agree switching the swiftrun/swiftconfig to using bash may not be the
> best idea due to the above reasons David has mentioned. However I do not
> believe switching over to using Java is the solution. In the end swiftrun is
> calling swift which is a shell script. Having a java process call a shell
> script which in turns starts java processes may not be the best plan of
> action. May I suggest using python? Not much changes between python versions
> and it has very good xml libraries to work with. I do not believe there will
> be very many comparability issues in using python but in all programming
> there are always these issues. Python is also much easier to read in my
> humble opinion.
>
> ----- Reply message -----
> From: "David Kelly" <dk0966 at cs.ship.edu>
> Date: Tue, Feb 1, 2011 11:24 pm
> Subject: [Swift-devel] Swiftconfig merge / changes
> To: "Michael Wilde" <wilde at mcs.anl.gov>
> Cc: "swift-devel" <swift-devel at ci.uchicago.edu>
>
>
>
> On Tue, Feb 1, 2011 at 12:07 PM, Michael Wilde <wilde at mcs.anl.gov> wrote:
>
> Also, Im very eager to reconcile and unify into one coordinated plan the
>> idea of merging the best aspects of Justin's ad-hoc configuration methid,
>> which he's documented on the SWFT wiki (see eg CoasterCookbook etc) and your
>> swiftconfig/swiftrun mechanism.
>>
>> I like the fact that Justin's is based on sh rather then perl - that will
>> make maintenance easier and reduce portability issues. But I also want to
>> bring back various concepts from swiftconfig/run into this mechanism.  If we
>> can agree on a spec, perhaps that would be a good project for you to
>> (resume) work on?
>>
>
> Here is what I see as the list of features we would need to add to the
> shell script in order to merge features:
>
> From swiftconfig:
>
> - XML editor. It should have some basic knowledge of valid inputs (for
> example, when you're editing the execution provider setting, it will only
> allow you to select one that swift knows)
> - Editor for adding/removing/modifying apps to the tc file and verifying
> formatting
> - Using templates to generate new site configurations
> - Using templates to generate new templates
> - Importing existing configuration files into the template format/directory
> structure
> - Creating and managing ssh configurations for various hosts
>
> From swiftrun:
>
> - A new swiftrun mechanism which reads from a template directory, generates
> the run.XXXX directory, links input data, and runs swift
> - The concept of site groups. For example, creating a group called
> "MCS-coasters" which combines the individual configurations of
> thrash-coasters, thwomp-coasters, etc.
> - Similar to above, application sets for handling different sets of apps
>
> It's possible we may not need or want all of these. I just wanted to point
> out what would be involved if we merged the main features as they currently
> exist in the swiftconfig/swiftrun utilities. I also have a list of
> suggestions for future improvements which include things like revision
> control for modifications and a tagging/search system for searching through
> previous swift runs.
>
> The suggestion was moving away from perl due to portability issues. I can
> understand that, but what do you think about doing it in Java? Bash is nice,
> but in some ways it suffers from the same portability issues that perl does.
> Here are a few issues I have run into so far. The "swift" shell script we
> use calls /bin/sh. Sometimes this is bash. On some linux systems it's
> actually a shell called ash, which is similar but does not always
> compatible. If you can assume there will always be a /bin/bash, there are
> individual differences between bash versions. When I was testing changes to
> usage stats, I ran into a really old version of bash which caused it to
> fail. Then there are things like /dev/udp which may or may not be turned on
> based on bash compilation options. Bash is nice, but it's a little limited
> in what is can do by itself without relying on a lot of external
> applications.. which may not be always be installed on the system or work as
> expected.
>
> The nice thing about doing it in Java is that you can bank on Java 1.5 or
> later being there regardless of whatever system quirks you may run into.
> Java also includes XML handling libraries and has nice regular expression
> system which would make this a little easier to handle. It will also be
> easier to manage as complexity increases in the future.
>
> Shifting gears a bit.. what I would _really_ would like in the future is a
> graphical interface on top of swiftconfig and swiftrun. I may be the only
> one, but I haven't given up on this idea :-)
>
> - Have one visual interface running locally which will let me do everything
> - Easily and visually configure swift to submit jobs on a remote system
> - Visually create a workflow which generates a swift script
> - Select a swift script and be able to easily change parameters, queue,
> maxtime, # nodes, etc on the fly
> - See a visual progression of what is happening. Something like a swing
> version of -tui
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110202/63af4964/attachment.html>