[Swift-devel] Next steps on SwiftR - re Fwd: [Swift-user] Do you have any resource for learning about SwiftR?

Fri Sep 9 08:44:42 CDT 2011

Have you looked at the Technology Insertion Service (TIS) in XSEDE?  This would be a way to push Swift (SwiftR?) forward there.

Dan

On Sep 9, 2011, at 9:41 AM, Michael Wilde wrote:

> Comments inline...
> 
> ----- Original Message -----
>> From: "Tim Armstrong" <tga at uchicago.edu>
>> Subject: Re: Next steps on SwiftR - re Fwd: [Swift-user] Do you have any resource for learning about SwiftR?
>> ...
>> I'm going to have a look at the CRAN docs. It sounds like it won't
>> meet CRAN standards as it requires a pre-built version of swift to be
>> bundled. I can look into what it would take to get it up to spec
>> there.
> 
> It should be feasible to do an svn co of Swift source and build it to create the Swift dist from within the SwiftR build. Would that pass CRAN requirements? Only way to find out I think is to test it and then submit to CRAN.
> 
>> I think we only have two serious sources of documentation: the wiki
>> page and the R docs. My website was only really intended to show the
>> benchmarks. I've removed the installation instructions from that page,
>> so there is now no real documentation on there. I agree it would make
>> sense to consolidate the documentation. Maybe the most sensible
>> approach would be to have the R help docs be the canonical version,
>> and generate a web version of those docs that would be released at the
>> same time as new versions of the package.
> 
> I agree - that sounds best. Lets post it on the Swift web and also link there from the OpenMx wiki.
> 
>> Then all the authoritative documentation (including Quickstart, FAQ,
>> Troubleshooting, etc) would be consolidated in that place.
> 
> Yes, that seems the best approach.
> 
>> Any
>> supplemental documentation that doesn't really make sense to include
>> in the CRAN package, e.g. about running SwiftR on particular clusters,
>> could then remain on the SWFT wiki.
> 
> Lets move that to asciidoc within the Swift web.
> 
>> I agree with the additional sites. Beagle we should get going, and
>> having good support for Teragrid and OSG makes a lot of sense. I'm
>> still not sure what the best way to manage custom Swift configs would
>> be. I think that leaving users to hand-write configs from scratch is
>> too error-prone. Good documentation + a bunch of working examples is
>> probably the most realistic solution, but it would be nice to allow a
>> wider set of configurations to be used directly from R without editing
>> swift configs. The one thing that springs to mind now is automatic
>> resource provisioning: I have that working with PBS, but not with the
>> full range of clusters
> 
> We should integrate this with Swift gensites which David is developing to make swift configuration easier.
> 
> - Mike
> 
>> 
>> - Tim
>> 
>> ---- Original message ----
>>> Date: Tue, 30 Aug 2011 10:34:39 -0500 (CDT)
>>> From: Michael Wilde <wilde at mcs.anl.gov>
>>> Subject: Next steps on SwiftR - re Fwd: [Swift-user] Do you have any
>>> resource for learning about SwiftR?
>>> To: Tim Armstrong <tga at uchicago.edu>, David Kelly
>>> <davidk at ci.uchicago.edu>
>>> Cc: swift-devel Devel <swift-devel at ci.uchicago.edu>, Lorenzo Pesce
>>> <lpesce at uchicago.edu>
>>> 
>>> David, can you schedule some time to meet with Tim, learn and try
>>> SwiftR, and work to publicize it on the Swift web?
>>> 
>>> Tim, do you think its ready to submit to CRAN? Does it meet the CRAN
>>> install criteria wrt building all included packages from source? (Ie,
>>> Swift?)
>>> 
>>> How should we handle the docs issue? I think we need to consolidate
>>> docs from the at least 3 sources I know about and get them into both
>>> the R help and echo them into an asciidoc page on the Swift web?
>>> (SwiftR R help; Tim's page; SWFT pages(2); also, do we still have
>>> docs on it on the OpenMx wiki? Last I looked the Swift help page
>>> there now refers back to SWFT pages).
>>> 
>>> Lastly, what do we need to do for additional SwiftR site support?
>>> - config and test on beagle
>>> - integrate with more swift configs
>>> - test and support for TG/XSEDE and OSG
>>> 
>>> Thanks,
>>> 
>>> - Mike
>>> 
>>> 
>>> ----- Forwarded Message -----
>>> From: "Michael Wilde" <wilde at mcs.anl.gov>
>>> To: "Lorenzo Pesce" <lpesce at uchicago.edu>
>>> Cc: "Tim Armstrong" <tga at uchicago.edu>
>>> Sent: Tuesday, August 30, 2011 10:22:06 AM
>>> Subject: Re: [Swift-user] Do you have any resource for learning about
>>> SwiftR?
>>> 
>>> Lorenzo,
>>> 
>>> The SwiftR documentation is currently at:
>>> 
>>> http://www.ci.uchicago.edu/wiki/bin/view/SWFT/SwiftR
>>> 
>>> which also provides a quick start guide at:
>>> 
>>> http://www.ci.uchicago.edu/wiki/bin/view/SWFT/SwiftRQuickstart
>>> 
>>> Further examples and some performance measurements are at:
>>> 
>>> http://people.cs.uchicago.edu/~tga/swiftR/
>>> 
>>> And more examples are available with ?SwiftR help once you load the
>>> package:
>>>> source("http://people.cs.uchicago.edu/~tga/swiftR/getSwift.R")
>>> 
>>> I just built an R-2.13.1 release on Beagle with plain gcc, which I
>>> think *should* be runnable in parallel on worker nodes. (Not yet
>>> tested though). This R should be capable of running SwiftR. Im hoping
>>> that Tim cam verify this soon. We'll likely need an additional SwiftR
>>> server name and config for Beagle and other Cray systems.
>>> 
>>> We'll try to consolidate the SwiftR documentation in a user guide on
>>> the Swift in the future. Tim, can you do a quick check of the
>>> documentation to make sure its still correct and that it points to
>>> the latest SwiftR package?
>>> 
>>> Thanks,
>>> 
>>> - Mike
>>> 
>>> 
>>> ----- Original Message -----
>>>> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
>>>> To: swift-user at ci.uchicago.edu
>>>> Sent: Tuesday, August 30, 2011 9:22:51 AM
>>>> Subject: [Swift-user] Do you have any resource for learning about
>>>> SwiftR?
>>>> Hi -
>>>> 
>>>> I want to run relatively small sized simulations (say at most 50
>>>> cores
>>>> or so, probably mostly one or two) but many many times over. The
>>>> simulations will be coded in R.
>>>> 
>>>> Thanks a lot!
>>>> 
>>>> Lorenzo
>>>> _______________________________________________
>>>> Swift-user mailing list
>>>> Swift-user at ci.uchicago.edu
>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>> 
>>> --
>>> Michael Wilde
>>> Computation Institute, University of Chicago
>>> Mathematics and Computer Science Division
>>> Argonne National Laboratory
>>> 
>>> --
>>> Michael Wilde
>>> Computation Institute, University of Chicago
>>> Mathematics and Computer Science Division
>>> Argonne National Laboratory
>>> 
> 
> -- 
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Daniel S. Katz
University of Chicago
(773) 834-7186 (voice)
(773) 834-6818 (fax)
d.katz at ieee.org or dsk at ci.uchicago.edu
http://www.ci.uchicago.edu/~dsk/