[Swift-devel] Next steps on SwiftR - re Fwd: [Swift-user] Do you have any resource for learning about SwiftR?

Fri Sep 9 08:49:20 CDT 2011

Thats an excellent idea Dan. David, can you explore this and report back on whats involved?

(We should still work to get SwiftR into CRAN as that's the expected source of all R packages).

Thanks,

- Mike

----- Original Message -----
> From: "Daniel S. Katz" <dsk at ci.uchicago.edu>
> To: "Michael Wilde" <wilde at mcs.anl.gov>
> Cc: "Tim Armstrong" <tga at uchicago.edu>, "Lorenzo Pesce" <lpesce at uchicago.edu>, "swift-devel Devel"
> <swift-devel at ci.uchicago.edu>
> Sent: Friday, September 9, 2011 8:44:42 AM
> Subject: Re: [Swift-devel] Next steps on SwiftR - re Fwd: [Swift-user] Do you have any resource for learning about
> SwiftR?
> Have you looked at the Technology Insertion Service (TIS) in XSEDE?
> This would be a way to push Swift (SwiftR?) forward there.
> 
> Dan
> 
> 
> On Sep 9, 2011, at 9:41 AM, Michael Wilde wrote:
> 
> > Comments inline...
> >
> > ----- Original Message -----
> >> From: "Tim Armstrong" <tga at uchicago.edu>
> >> Subject: Re: Next steps on SwiftR - re Fwd: [Swift-user] Do you
> >> have any resource for learning about SwiftR?
> >> ...
> >> I'm going to have a look at the CRAN docs. It sounds like it won't
> >> meet CRAN standards as it requires a pre-built version of swift to
> >> be
> >> bundled. I can look into what it would take to get it up to spec
> >> there.
> >
> > It should be feasible to do an svn co of Swift source and build it
> > to create the Swift dist from within the SwiftR build. Would that
> > pass CRAN requirements? Only way to find out I think is to test it
> > and then submit to CRAN.
> >
> >> I think we only have two serious sources of documentation: the wiki
> >> page and the R docs. My website was only really intended to show
> >> the
> >> benchmarks. I've removed the installation instructions from that
> >> page,
> >> so there is now no real documentation on there. I agree it would
> >> make
> >> sense to consolidate the documentation. Maybe the most sensible
> >> approach would be to have the R help docs be the canonical version,
> >> and generate a web version of those docs that would be released at
> >> the
> >> same time as new versions of the package.
> >
> > I agree - that sounds best. Lets post it on the Swift web and also
> > link there from the OpenMx wiki.
> >
> >> Then all the authoritative documentation (including Quickstart,
> >> FAQ,
> >> Troubleshooting, etc) would be consolidated in that place.
> >
> > Yes, that seems the best approach.
> >
> >> Any
> >> supplemental documentation that doesn't really make sense to
> >> include
> >> in the CRAN package, e.g. about running SwiftR on particular
> >> clusters,
> >> could then remain on the SWFT wiki.
> >
> > Lets move that to asciidoc within the Swift web.
> >
> >> I agree with the additional sites. Beagle we should get going, and
> >> having good support for Teragrid and OSG makes a lot of sense. I'm
> >> still not sure what the best way to manage custom Swift configs
> >> would
> >> be. I think that leaving users to hand-write configs from scratch
> >> is
> >> too error-prone. Good documentation + a bunch of working examples
> >> is
> >> probably the most realistic solution, but it would be nice to allow
> >> a
> >> wider set of configurations to be used directly from R without
> >> editing
> >> swift configs. The one thing that springs to mind now is automatic
> >> resource provisioning: I have that working with PBS, but not with
> >> the
> >> full range of clusters
> >
> > We should integrate this with Swift gensites which David is
> > developing to make swift configuration easier.
> >
> > - Mike
> >
> >>
> >> - Tim
> >>
> >> ---- Original message ----
> >>> Date: Tue, 30 Aug 2011 10:34:39 -0500 (CDT)
> >>> From: Michael Wilde <wilde at mcs.anl.gov>
> >>> Subject: Next steps on SwiftR - re Fwd: [Swift-user] Do you have
> >>> any
> >>> resource for learning about SwiftR?
> >>> To: Tim Armstrong <tga at uchicago.edu>, David Kelly
> >>> <davidk at ci.uchicago.edu>
> >>> Cc: swift-devel Devel <swift-devel at ci.uchicago.edu>, Lorenzo Pesce
> >>> <lpesce at uchicago.edu>
> >>>
> >>> David, can you schedule some time to meet with Tim, learn and try
> >>> SwiftR, and work to publicize it on the Swift web?
> >>>
> >>> Tim, do you think its ready to submit to CRAN? Does it meet the
> >>> CRAN
> >>> install criteria wrt building all included packages from source?
> >>> (Ie,
> >>> Swift?)
> >>>
> >>> How should we handle the docs issue? I think we need to
> >>> consolidate
> >>> docs from the at least 3 sources I know about and get them into
> >>> both
> >>> the R help and echo them into an asciidoc page on the Swift web?
> >>> (SwiftR R help; Tim's page; SWFT pages(2); also, do we still have
> >>> docs on it on the OpenMx wiki? Last I looked the Swift help page
> >>> there now refers back to SWFT pages).
> >>>
> >>> Lastly, what do we need to do for additional SwiftR site support?
> >>> - config and test on beagle
> >>> - integrate with more swift configs
> >>> - test and support for TG/XSEDE and OSG
> >>>
> >>> Thanks,
> >>>
> >>> - Mike
> >>>
> >>>
> >>> ----- Forwarded Message -----
> >>> From: "Michael Wilde" <wilde at mcs.anl.gov>
> >>> To: "Lorenzo Pesce" <lpesce at uchicago.edu>
> >>> Cc: "Tim Armstrong" <tga at uchicago.edu>
> >>> Sent: Tuesday, August 30, 2011 10:22:06 AM
> >>> Subject: Re: [Swift-user] Do you have any resource for learning
> >>> about
> >>> SwiftR?
> >>>
> >>> Lorenzo,
> >>>
> >>> The SwiftR documentation is currently at:
> >>>
> >>> http://www.ci.uchicago.edu/wiki/bin/view/SWFT/SwiftR
> >>>
> >>> which also provides a quick start guide at:
> >>>
> >>> http://www.ci.uchicago.edu/wiki/bin/view/SWFT/SwiftRQuickstart
> >>>
> >>> Further examples and some performance measurements are at:
> >>>
> >>> http://people.cs.uchicago.edu/~tga/swiftR/
> >>>
> >>> And more examples are available with ?SwiftR help once you load
> >>> the
> >>> package:
> >>>> source("http://people.cs.uchicago.edu/~tga/swiftR/getSwift.R")
> >>>
> >>> I just built an R-2.13.1 release on Beagle with plain gcc, which I
> >>> think *should* be runnable in parallel on worker nodes. (Not yet
> >>> tested though). This R should be capable of running SwiftR. Im
> >>> hoping
> >>> that Tim cam verify this soon. We'll likely need an additional
> >>> SwiftR
> >>> server name and config for Beagle and other Cray systems.
> >>>
> >>> We'll try to consolidate the SwiftR documentation in a user guide
> >>> on
> >>> the Swift in the future. Tim, can you do a quick check of the
> >>> documentation to make sure its still correct and that it points to
> >>> the latest SwiftR package?
> >>>
> >>> Thanks,
> >>>
> >>> - Mike
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
> >>>> To: swift-user at ci.uchicago.edu
> >>>> Sent: Tuesday, August 30, 2011 9:22:51 AM
> >>>> Subject: [Swift-user] Do you have any resource for learning about
> >>>> SwiftR?
> >>>> Hi -
> >>>>
> >>>> I want to run relatively small sized simulations (say at most 50
> >>>> cores
> >>>> or so, probably mostly one or two) but many many times over. The
> >>>> simulations will be coded in R.
> >>>>
> >>>> Thanks a lot!
> >>>>
> >>>> Lorenzo
> >>>> _______________________________________________
> >>>> Swift-user mailing list
> >>>> Swift-user at ci.uchicago.edu
> >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >>>
> >>> --
> >>> Michael Wilde
> >>> Computation Institute, University of Chicago
> >>> Mathematics and Computer Science Division
> >>> Argonne National Laboratory
> >>>
> >>> --
> >>> Michael Wilde
> >>> Computation Institute, University of Chicago
> >>> Mathematics and Computer Science Division
> >>> Argonne National Laboratory
> >>>
> >
> > --
> > Michael Wilde
> > Computation Institute, University of Chicago
> > Mathematics and Computer Science Division
> > Argonne National Laboratory
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> --
> Daniel S. Katz
> University of Chicago
> (773) 834-7186 (voice)
> (773) 834-6818 (fax)
> d.katz at ieee.org or dsk at ci.uchicago.edu
> http://www.ci.uchicago.edu/~dsk/

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory