[Swift-devel] Swiftconfig merge / changes

Wed Feb 2 15:34:56 CST 2011

On Wed, 2011-02-02 at 15:31 -0600, Jonathan Monette wrote:
> I was not aware that a java program could call another java programs 
> main method.  By not aware I mean I do not know how.

The main method is a plain static method. So ClassName.main(new String[]
{"arg1", "arg2", ...});

> 
> On 2/2/11 3:26 PM, Mihael Hategan wrote:
> > On Wed, 2011-02-02 at 13:22 -0800, Mihael Hategan wrote:
> >> On Wed, 2011-02-02 at 00:04 -0600, jon.monette at gmail.com wrote:
> >>> I agree switching the swiftrun/swiftconfig to using bash may not be
> >>> the best idea due to the above reasons David has mentioned. However I
> >>> do not believe switching over to using Java is the solution. In the
> >>> end swiftrun is calling swift which is a shell script. Having a java
> >>> process call a shell script which in turns starts java processes may
> >>> not be the best plan of action.
> >> That doesn't need to be so. A java program can invoke the main() method
> >> of swift.
> > In fact, this may very well turn out to fit into that swift shell model
> > that Mike mentioned that would keep coaster workers (and other JVM
> > persistent things) alive between swift runs.
> >
> >>>   May I suggest using python? Not much changes between python versions
> >>> and it has very good xml libraries to work with. I do not believe
> >>> there will be very many comparability issues in using python but in
> >>> all programming there are always these issues. Python is also much
> >>> easier to read in my humble opinion.
> >>>
> >>> ----- Reply message -----
> >>> From: "David Kelly"<dk0966 at cs.ship.edu>
> >>> Date: Tue, Feb 1, 2011 11:24 pm
> >>> Subject: [Swift-devel] Swiftconfig merge / changes
> >>> To: "Michael Wilde"<wilde at mcs.anl.gov>
> >>> Cc: "swift-devel"<swift-devel at ci.uchicago.edu>
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Feb 1, 2011 at 12:07 PM, Michael Wilde<wilde at mcs.anl.gov>
> >>> wrote:
> >>>
> >>>          Also, Im very eager to reconcile and unify into one
> >>>          coordinated plan the idea of merging the best aspects of
> >>>          Justin's ad-hoc configuration methid, which he's documented on
> >>>          the SWFT wiki (see eg CoasterCookbook etc) and your
> >>>          swiftconfig/swiftrun mechanism.
> >>>
> >>>          I like the fact that Justin's is based on sh rather then perl
> >>>          - that will make maintenance easier and reduce portability
> >>>          issues. But I also want to bring back various concepts from
> >>>          swiftconfig/run into this mechanism.  If we can agree on a
> >>>          spec, perhaps that would be a good project for you to (resume)
> >>>          work on?
> >>>
> >>> Here is what I see as the list of features we would need to add to the
> >>> shell script in order to merge features:
> >>>
> >>>  From swiftconfig:
> >>>
> >>> - XML editor. It should have some basic knowledge of valid inputs (for
> >>> example, when you're editing the execution provider setting, it will
> >>> only allow you to select one that swift knows)
> >>> - Editor for adding/removing/modifying apps to the tc file and
> >>> verifying formatting
> >>> - Using templates to generate new site configurations
> >>> - Using templates to generate new templates
> >>> - Importing existing configuration files into the template
> >>> format/directory structure
> >>> - Creating and managing ssh configurations for various hosts
> >>>
> >>>  From swiftrun:
> >>>
> >>> - A new swiftrun mechanism which reads from a template directory,
> >>> generates the run.XXXX directory, links input data, and runs swift
> >>> - The concept of site groups. For example, creating a group called
> >>> "MCS-coasters" which combines the individual configurations of
> >>> thrash-coasters, thwomp-coasters, etc.
> >>> - Similar to above, application sets for handling different sets of
> >>> apps
> >>>
> >>> It's possible we may not need or want all of these. I just wanted to
> >>> point out what would be involved if we merged the main features as
> >>> they currently exist in the swiftconfig/swiftrun utilities. I also
> >>> have a list of suggestions for future improvements which include
> >>> things like revision control for modifications and a tagging/search
> >>> system for searching through previous swift runs.
> >>>
> >>> The suggestion was moving away from perl due to portability issues. I
> >>> can understand that, but what do you think about doing it in Java?
> >>> Bash is nice, but in some ways it suffers from the same portability
> >>> issues that perl does. Here are a few issues I have run into so far.
> >>> The "swift" shell script we use calls /bin/sh. Sometimes this is bash.
> >>> On some linux systems it's actually a shell called ash, which is
> >>> similar but does not always compatible. If you can assume there will
> >>> always be a /bin/bash, there are individual differences between bash
> >>> versions. When I was testing changes to usage stats, I ran into a
> >>> really old version of bash which caused it to fail. Then there are
> >>> things like /dev/udp which may or may not be turned on
> >>> based on bash compilation options. Bash is nice, but it's a little
> >>> limited in what is can do by itself without relying on a lot of
> >>> external applications.. which may not be always be installed on the
> >>> system or work as expected.
> >>>
> >>> The nice thing about doing it in Java is that you can bank on Java 1.5
> >>> or later being there regardless of whatever system quirks you may run
> >>> into. Java also includes XML handling libraries and has nice regular
> >>> expression system which would make this a little easier to handle. It
> >>> will also be easier to manage as complexity increases in the future.
> >>>
> >>> Shifting gears a bit.. what I would _really_ would like in the future
> >>> is a graphical interface on top of swiftconfig and swiftrun. I may be
> >>> the only one, but I haven't given up on this idea :-)
> >>>
> >>> - Have one visual interface running locally which will let me do
> >>> everything
> >>> - Easily and visually configure swift to submit jobs on a remote
> >>> system
> >>> - Visually create a workflow which generates a swift script
> >>> - Select a swift script and be able to easily change parameters,
> >>> queue, maxtime, # nodes, etc on the fly
> >>> - See a visual progression of what is happening. Something like a
> >>> swing version of -tui
> >>>
> >>>
> >>> _______________________________________________
> >>> Swift-devel mailing list
> >>> Swift-devel at ci.uchicago.edu
> >>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >