[Swift-devel] Re: swiftconfig

[Arjun Comar]

Hey David,
I'd love to see the source for this, is it in svn? If you'd like, I can
write the ncurses frontend (I think that's what you meant, correct me if I'm
wrong) while I work on the various quickset profiles functional. Just in
case you haven't been filled in yet, Mike's having me work on creating a few
quickset profiles for swiftconfig, the idea being that sometimes its just
easier to drop in a working set of configurations and tweaking from there
rather than starting from scratch every time. So far here are the profiles
Mike came up with, and I've been working on:
1) Remote execution via ssh, single site
2) Local execution via PBS, single site
3) Local execution via Coasters+PBS, single site
4) Remote execution via Coasters+SSH:PBS, single site

Templates for the sites.xml files for profiles 1-3 are ready, tested, and
functional. I'm having issues getting 4 to work (see my correspondence with
Mike in swift-devel), but that should hopefully get resolved today. After
that, my major priority is to create and test a variety of multi-site
profiles. The variety for these profiles is primarily going to come from the
number of sites, the coaster settings, the specific scheduler used at each
site, etc.. I don't have any particular thoughts on the specifics for these
profiles, but I'll be working on that today.

In any case, let me know if I can help with anything.

Arjun


> Hello all,
>
> I am working on a utility to modify configuration files called swiftconfig.
> This is still in the early stages, so there is a lot of room for changes
> and
> new ideas. I believe there is some overlap between this project and what
> some other students will be doing this summer, so if anyone would like to
> work with me on this, please feel free.
>
> I envision swiftconfig as a simple text-based configuration program. It
> will
> be written in Perl and use the curses library for easier editing. It should
> hopefully make swift configuration a little easier and prevent silly
> mistakes like typos in xml which could keep swift from running.  Everything
> that can be done within the editor should also be able to be done directly
> from the command line. This should make it easier to expand upon in the
> future. For example, a web or GUI based application could be written fairly
> quickly that would only need to call swiftconfig with the correct command
> line options.
>
> There are three files swiftconfig can modify: tc.data, sites.xml, and
> auth.defaults.
>
> The options for transformation mode include
>
>    -host        # Host name
>    -name      # Translation name
>    -path        # Path to executable
>    -profile     # Profile arguments, defaults to null
>    -tcfile       # Location of tc file. If not specified, find tc.data
> based on location of swift
>    -overwrite # If a duplicate is found, overwrite the old entry without
> prompting
>
> Since platform and installation status are no longer used, they will
> default
> to INTEL32::LINUX and INSTALLED.
> Here is an example of swiftconfig in transformation mode.
>
> $ swiftconfig -host localhost -name wc -path /usr/bin/wc
>
> tc.data should then have the line:
> localhost    wc    /usr/bin/wc    INSTALLED    INTEL32::LINUX    null
>
> If there is already an entry with the name wc, it should prompt the user to
> answer yes/no if the user wants to overwrite it (unless -overwrite is
> given)
>
> For sites.xml, swiftconfig should allow the user to use existing examples
> or
> specify their own. Here are the options:
>
>    -template       # Use existing commented example for defaults (skynet,
> teraport, etc)
>    -entry            # Name of new entry (pool handle)
>    -gridftp          # Specify gridftp url
>    -jobuniverse  # Specify jobmanager universe
>    -joburl           # Specify jobmanager url
>    -jobmajor      # Specify jobmanager major value
>    -jobminor      # Specify jobmanager minor value
>    -directory      # Work directory
>    -exprovider   # Execution provider
>    -exmanager  # Execution job manager
>    -exurl            # Execution url
>    -remove        # Remove (comment out) an entry from sites.xml
>
> So, for example suppose a user has the following entry in sites.xml by
> default:
>
> <!--
>  <pool handle="teraport" >
>    <gridftp  url="gsiftp://tp-grid1.uchicago.edu" />
>    <jobmanager universe="vanilla" url="
> tp-grid1.uchicago.edu/jobmanager-pbs"
> major="2" />
>    <workdirectory >/home/tiberius/scratch/SWIFT-
> WORK</workdirectory>
>  </pool>
> -->
>
> The command:
>
> $ swiftconfig -template teraport
>
> Which would uncomment that from sites.xml as is. The user could also modify
> just a part of it:
>
> $ swiftconfig -template teraport -directory /tmp
>
> That should modify only the workdirectory and leave everything else the
> same.
>
> To create your own config, use -entry instead of -template
>
> $ swiftconfig -entry mynetwork -gridftp ftp.foo -exprovider gt4 (.. and so
> on)
>
> The final mode of swiftconfig is for auth.log in ssh configurations.
>
>    -auth        # Set to auth mode
>    -sshhost    # Name of remote ssh host
>    -sshmode    # Either password or passphrase
>    -sshuser    # SSH username
>    -sshpassword    # SSH password
>    -sshpassphrase    # SSH passphrase
>    -sshkey        # Location of SSH key
>
> Any other ideas or suggestions on how swiftconfig should work are welcome.
>
> David
>
>
> --
Arjun Comar, Rose-Hulman '12
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20100607/25d7b794/attachment.html>